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SUMMARY 


In  1977,  nearly  seven  percent  of  the  Department  of 

Defense  Budget  was  allocated  to  the  area  of  intelligence  and 

2 

communications.  The  command  and  control  (C  ) function,  an 

integral  part  of  intelligence  and  communications,  provides 

the  structure  which  enables  the  National  Command  Authority 

(NCA)  the  President  and  Secretary  of  Defense,  to  exercise 

command  and  control  over  deployed  U.S.  forces  through  the 

2 

Joint  Chiefs  of  Staff  (JCS) . Essentially,  the  C function 
integrates  the  surveillance  and  reaction  activities  of 
deployed  U.S.  military  groups,  thereby  ensuring  a unified 
defense  force — a highly  complex  task  vital  to  the  nation's 
security.  If  the  C*"  capability  were  sufficiently  low,  the 
U.S.  armed  forces  would  not  function  satisfactorily  and  the 
nation's  defense  posture  would  be  seriously  jeopardized. 

o 

In  view  of  the  significance  of  the  C*"  function,  a per- 

2 

tinent  question  is,  "How  much  C capability  is  enough"?  It 

is  possible  to  attempt  to  answer  in  the  absolute,  that  is, 

2 

say,  enough  C capability  exists  when  the  United  States 

armed  forces  can  be  certain  of  countering  all  projected 

threats  and  crises.  Trying  to  answer  the  question  of  enough 
2 

C in  the  absolute  is  absurd,  however,  for  if  the  weapons 

systems  and  personnel  to  be  commanded  and  controlled  are  of 

2 

sufficiently  low  quality,  an  excellent  C system  will  be  of 

2 

little  value.  The  question  of  enough  C is  but  a part  of 
the  larger  problem  of  a force  capable  of  meeting  the  goal  of 
defending  the  nation  at  all  times. 

2 

In  order  to  assess  the  value  of  a particular  C capa- 
bility, the  cost  of  achieving  that  capability  must  be  weighed 
against  the  potential  benefits  of  allocating  funds  else- 
where. Examination  of  the  breakdown  of  the  Department  of 
Defense  Budget  would  prompt  a reasonable  inquiry:  Is  this 
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allocation  scheme  the  best  overall  scheme  for  meeting  the 

long-term  goal  of  defending  the  nation?  More  specifically, 

2 

is  the  seven-percent  allocation  for  C sufficient  to  meet 
the  need  in  that  area?  The  answer  necessarily  involves 
assessing  the  requirements  for  and  the  benefits  to  be  de- 
rived from  expenditures  in  each  of  the  specified  DoD  program 

areas.  The  procedures  and  recommendations  discussed  with 
2 

respect  to  C generalize  trade-offs  among  objectives  on 
goals  at  any  level  of  generality  chosen  for  analysis. 

The  objective,  then,  is  to  determine,  given  the  fixed 

budget  size,  whether  the  chosen  dollar  allocations  are 

optimal.  The  word  "optimal"  implies  the  best  with  respect 

to  some  cost/benefit  criterion;  and  when  cost  is  fixed, 

optimal  means  the  option  with  highest  value.  Given  a fixed 

set  of  conditions  and  other  necessary  information,  the  value 

to  be  attached  to  a particular  option  can  be  established. 

Part  of  the  methodology  discussed  in  this  report  addresses 

that  question,  establishing  the  value  of  particular  levels 
2 

of  C capability  given  a specified  set  of  conditions. 

From  a decision  analysis  point  of  view,  this  problem 

involves  assigning  a value  to  the  action  of  deciding  to 

2 

deploy  a particular  C system  in  an  uncertain  future.  A 

large  decision  tree  would  be  constructed  to  accommodate  all 

2 

potentially  reasonable  C deployments,  the  uncertain  events, 
and  the  possible  outcomes.  After  the  assessment  of  proba- 
bilities and  values,  the  tree  would  then  be  folded  back 

2 

(evaluated)  to  yield  an  "optimal"  C decision.  The  problem 
in  such  a case  is  that  the  decision  tree  is  too  large  ("a 
very  bushy  mess,"  to  borrow  a Howard  Raiffa  metaphor),  and 
meaningful  probabilities  and  values  cannot  be  assessed. 

The  structuring  of  decision  trees  and  related  value 
assessments  are  discussed  in  several  sources,  among  them 
Raiffa  (1968)  and  Keeney  and  Raiffa  (1976).  The  use  of 


iii 


scenarios  as  an  aid  in  the  "bushy  mess"  problem  has  been 
discussed  in  O'Connor  and  Edwards  (1976) . This  report 
attempts  to  focus  the  foregoing  material  on  the  problem  at 
hand. 


The  discussion  of  the  problem  is  divided  into  four 
parts.  Section  2.0  deals  with  the  theory  of  assessing  value 
for  multiple  attributed  alternatives.  For  example,  the 
expenditures  in  each  area  of  the  defense  budget  can  be 
viewed  as  attributes  of  the  budget.  The  level  of  capability 
that  would  result  from  a particular  expenditure  is  an  uncer- 
tain alternative  which  itself  has  many  attributes.  Certain 

2 

desirable  capabilities  of  the  C are  attributes  of  the 
2 

overall  C capability  that  can  be  attained  for  an  expendi- 
ture of  Finding  the  most  preferred  system  that  can  be 

had  for  involves  the  assessment  of  trade-offs  among  the 

multiple  attributes  that  characterize  the  system. 

Section  3.0  discusses  the  use  of  scenarios  as  a solu- 
tion to  the  so  called  "bushy  mess"  problem.  The  assignment 
of  value  to  a multiple  attributed  option  is  dependent  on 
adequate  specification  of  the  situation  in  which  the  option 
will  be  deployed.  The  use  of  scenarios  facilitates  the 
representation  of  that  future.  This  approach  involves 
characterizing  each  system  as  a multi-attributed  alternative 
where  the  performance  of  the  system  in  the  scenario  is  one 
of  the  system  attributes.  The  value  of  the  system  in  each 
scenario  is  weighted  by  the  scenario  importance,  and  these 
weighted  values  are  summed  across  scenarios  to  yield  an 
overall  system  value. 

Section  4.0  illustrates  the  application  of  value  assess- 
ment procedures  to  a hypothetical  problem  of  evaluating 
alternative  architectural  candidates  for  a World  Wide  Mili- 
tary Command  and  Control  System  (WWMCCS) . The  analysis  was 
performed  to  determine  the  relative  value  of  potential 
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WWMCCS  crisis  requirements.  This  discussion  demonstrates  a 
workable  methodology  that  is  applicable  for  the  quantifica- 
tion of  the  value  of  requirements  and  the  optimization  of 
system  design. 

Section  5.0  further  illustrates  the  application  of 

2 

utility  assessment  procedures  to  the  C problem  by  using  a 
specific  problem.:  the  trade-offs  between  cost  and  multiple 

performance  measures.  This  discussion  emphasizes  the  neces- 
sity to  find  a performance  measure  that  can  be  interpreted 
in  terms  of  dollars.  There  is  obviously  no  magic  solution 
to  the  problem  of  comparing  cost  and  benefit  where  benefit 
cannot  be  translated  directly  into  dollars.  The  best 
approach  is  to  find  at  least  one  scenario  in  which  cost  and 
performance  could  be  traded  off.  Given  that  ability,  then 
the  trade-offs  between  performances  across  all  scenarios 
would  allow  translation  into  dollars  for  all  scenarios. 

This  report  covers  many  aspects  of  utility  analysis. 

2 

The  question  of  "how  much  C is  enough?"  is  addressed  at  a 
fairly  abstract  methodological  level,  and  emphasis  is  on  the 
methodology  that  might  be  used  to  answer  it.  The  question, 
though  appearing  to  be  an  absolute  one,  is  necessarily 
relative,  involving  multi-criteria  trade-offs.  Most  of  the 
report  deals  with  the  methodology  for  making  such  trade- 
offs. That  methodology  is  well  established,  and,  as  exem- 
plified in  Section  4.0,  it  has  been  applied  repeatedly  to 
practical  problems. 
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PROCEDURES  FOR  ASSESSING  THE  VALUE  OF 
COMMAND  AND  CONTROL  CAPABILITIES 


1.0  INTRODUCTION 

Of  the  112.7  billion  dollars  allocated  for  the  1977 

Department  of  Defense  Budget,  the  area  of  intelligence  and 

communications  received  7.7  billion  dollars,  nearly  seven 

percent  of  the  entire  defense  budget.^  An  integral  part  of 

intelligence  and  communications  is  the  command  and  control 

(C^)  function  (also  called  C^--command,  control,  and  communi- 

cations) . The  national  C structure  provides  for  the 

ability  of  the  National  Command  Authority  (NCA) , the  President 

and  Secretary  of  Defense,  to  exercise  command  and  control 

over  deployed  U.S.  forces  through  the  Joint  Chiefs  of  Staff 

(JCS) . Various  levels  of  C^  exist,  and,  in  fact,  a precise 

definition  of  C^  is  difficult  to  provide.  A simple  view  of 

U.S.  defenses  is  that  they  involve  three  necessary  functions. 

One  is  surveillance  for  the  purpose  of  ascertaining  the 

capabilities  and  status  of  enemy  forces.  A second  function 

is  reaction.  U.S.  forces  must  be  able  to  react  appropriately 

2 

to  various  levels  of  threats.  A third  function,  C , integrates 
the  surveillance  and  reaction  functions  and  provides  for  a 
unified  defense  force.  Although  there  exist  many  more 
aspects  of  C^  than  this  simple  discussion  indicates,  it  is 
safe  to  say  that  if  the  U.S.  C^  capability  were  sufficiently 
low,  the  U.S.  armed  forces  would  not  function  satisfactorily 
in  response  to  a threat,  and  the  nation  would  be  unable  to 
defend  itself  adequately.  In  light  of  this,  a pertinent 
question  is,  "How  much  capability  is  enough?"  The  answer, 
among  other  things,  depends  on  the  meaning  of  the  term 
"enough. " 


^D.  H.  Rumsfeld,  FY  1977  Report  of  the  Secretary  of  Defense 
(United  States  Department  of  Defense,  1976) , pi  559. 
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It  is  possible  to  attempt  to  answer  this  question  in 
the  absolute,  that  is,  say,  enough  capability  exists  when 
the  United  States  armed  forces  can  be  certain  of  countering 
all  projected  threats  and  crises.  But  what  does  "certain" 
mean?  Does  "certain"  mean  that  all  threats  will  be  countered 
with  probability  1.0,  .99,  .90,  or  the  like?  And  who  is  to 
assess  these  probabilities?  If  a clairvoyant  who  could 
foresee  the  consequences  o^  current  'decisions  existed,  that 
clairvoyant  could  be  used  to  aid  in  designing  a system. 
Obviously,  however,  expert  judgment  is  all  that  is  available 
to  substitute  for  the  clairvoyance  that  would  provide  perfect 
and  certain  answers. 

Trying  to  answer  the  question  of  enough  in  the 
absolute  is,  however,  absurd,  for  if  the  weapons  systems  and 
personnel  to  be  commanded  and  controlled  are  of  sufficiently 
low  quality,  an  excellent  system  will  be  of  little  value. 
This  correctly  implies  that  the  value  to  be  attached  to  a 
particular  level  of  capability  is  dependent  upon  capabilitie 
in  weapons,  personnel,  and  the  like. 

The  question  of  enough  is  but  a part  of  the  larger 
problem  of  a force  capable  of  meeting  the  goal  of  defending 
the  nation  at  all  times.  This  goal  is  in  turn  part  of  an 
even  higher  goal  of  ensuring  the  security,  freedom,  and 
social  benefits  necessary  for  long-term  sustenance  of  the 
union.  The  question  is,  therefore,  one  involving  trade-offs 
among  multiple  objectives,  and  it  is  in  this  light  that  the 
question  will  be  addressed. 

In  order  to  assess  the  value  of  a particular  capa- 
bility, the  cost  of  achieving  that  capability  must  be  weighed 
against  the  potential  benefits  of  allocating  funds  elsewhere. 
The  discussion  will  be  confined  to  benefits  derived  from  the 

defense  of  the  nation.  The  procedures  and  recommendations 

2 

to  be  discussed  with  respect  to  C generalize  trade-offs 
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among  objectives  or  goals  at  any  level  of  generality  chosen 
for  analysis. 

One  breakdown  of  the  1977  Department  of  Defense  Budget, 
a financial  sumjnary  by  billions  of  dollars,  lists  the  following 
categories  and  associated  1977  allocations . ^ 


MILITARY  PROGRAM 


ALLOCATION 

(billions  of  dollars) 


Strategic  Forces  9.4 
General  Purpose  Forces  40.2 
Intelligence  and  Communications  7.7 
Airlift  and  Sealift  1.6 
Guard  and  Reserve  Forces  5.9 
Research  and  Development  10.5 
Central  Supply  and  Maintenance  10.9 
Training,  Medical  and  Other  General 

Personnel  Activities  23.0 
Administration  and  Associated  Activities  2.1 
Support  to  Other  Nations  1.4 


112.7 


A reasonable  question,  "Is  this  allocation  scheme  the 
best  overall  scheme  for  meeting  the  long-term  goal  of  defending 
the  nation?"  may  be  asked.  More  specifically,  is  the  7.7 
billion  dollar  allocation  for  C^  sufficient  to  meet  the  need 
in  that  area?  The  answer  necessarily  involves  assessing 
the  requirements  for  and  benefits  to  be  derived  from  expendi- 
tures in  each  of  the  above  areas. 

An  important  related  question  involves  whether  or  not 
the  defense  budget  is  large  enough.  It  is  often  argued  that 
U.S.  defense  needs  are  not  being  met,  that  the  U.S.  is  only 
marginally  able  to  deter  Soviet  aggression,  and  the  like; 
this  argument  implies  either  more  defense  spending  or  more 
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efficient  use  of  allocated  funds,  or  both.  This  question 
will  not  be  addressed  here.  The  decision  on  the  magnitude 
of  the  defense  budget  as  compared  to  other  budgets  should  be 
made  by  considering  the  importance  of  the  goal  of  overall 
U.S.  defense  as  compared  to  other  high-level  goals.  The 
types  of  inter-goal  trade-offs  at  this  higher  level  will  be 
illustrated  by  the  simplified  approach  using  the  three 
defense  categories  chosen  for  discussion. 

The  question,  then,  is  whether,  given  the  fixed  budget 
size,  the  chosen  dollar  allocations  are  optimali*  The  word 
"optimal"  implies  the  best  with  respect  to  some  cost/benefit 
criterion,  and  when  cost  is  fixed,  optimal  means  the  option 
with  highest  value.  The  problem  of  determining  the  highest 
value  system,  given  a fixed  level  of  cost,  obviously  depends 
on  a specified  set  of  conditions. 

Such  conditions  are: 

the  time  period  under  consideration, 

the  nature  of  the  world  situation  during  that 
period, 

the  nature  of  the  enemy  threat  during  that 
period, 

the  status  of  the  U.S.  weapons  systems  during 
that  time,  and 

the  status  of  the  personnel  forces  during  that 
time . 

Given  this  and  any  other  necessary  information,  the 
value  to  be  attached  to  a particular  option  can  be  established. 
Part  of  the  methodology  to  be  discussed  addresses  that 
question,  establishing  the  value  of  particular  levels  of 
capability  given  a specified  set  of  conditions. 
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A decision  analyst  would  view  this  problem  as  one  of 
assigning  a value  to  the  action  of  deciding  to  deploy  a par- 
ticular system  in  an  uncertain  future.  If  a decision- 
tree  approach  were  used,  relevant  uncertain  events  (including 
the  decision  maker's  subsequent  actions  contingent  on  events) 
would  be  established,  and  a large  decision  tree  which  lays 
out  all  potentially  reasonable  deployments,  the  uncertain 
events,  and  the  possible  outcomes  would  be  constructed. 
Probabilities  and  values  would  be  assessed  and  inserted  into 
the  decision  tree,  which  would  then  be  folded  back  (evaluated) 
to  yield  an  "optimal"  system.  The  only  trouble  is  that 
the  decision  tree  is  usually  far  too  large.  There  are  so 
many  branches  containing  uncertain  events  and  consequences 
that  meaningful  probabilities  and  values  cannot  be  assessed. 

The  assignment  of  a value  to  a particular  system  corre- 
sponding to  a particular  level  of  cost  thus  depends  on  the 
evaluation  of  a large  decision  tree,  "a  very  bushy  mess"  (to 
borrow  a Howard  Raiffa  metaphor) . The  assignment  of  a value 
to  a particular  system  conditional  on  one  branch  of  the 
decision  tree  is  but  a small  part  of  the  larger  problem  of 
structuring  the  tree.  The  structuring  of  decision  trees  and 
related  value  assessments  are  discussed  in  several  sources, 
among  them  Raiffa  (1968)  and  Keeney  and  Raiffa  (1976).  The 
use  of  scenarios  as  an  aid  in  the  "bushy  mess"  situation  has 
been  discussed  in  O'Connor  and  Edwards  (1976).  This  report 
will  attempt  to  focus  that  material  on  the  problem  at  hand. 

The  discussion  will  be  divided  into  four  parts.  The 
first  part  will  deal  with  the  theory  of  assessing  value  for 
multiple  attributed  alternatives.  The  proposed  defense 
budget  is  but  one  of  an  infinite  number  that  could  be  adopted. 
The  expenditures  in  each  area  can  be  viewed  as  attributes  of 
the  budget.  The  level  of  capability  that  will  result  from 
the  expenditure  in  a particular  area  is  an  uncertain  alterna- 
tive which  itself  has  many  attributes.  Thus,  the  expenditure 
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of  $Xj^  on  a specific  plan  for  will  result  in  some  level 
of  capability  for  the  NCA  to  interact  with  other  heads  of 
state.  Similarly,  the  expenditure  will  lead  to  some  unknown 
level  of  capability  of  the  NCA  to  interface  with  the  World 
Wide  Military  Command  and  Control  System  (WWMCCS) . These 
capabilities  and  others  are  attributes  of  the  c2  capability 
that  can  be  attained  for  an  expenditure  of  $Xi.  One  problem 
is  to  find  the  most  preferred  system  that  can  be  had  for 
$Xi,  and  the  solution  necessarily  involves  the  assessment  of 
trade-offs  among  the  multiple  attributes  that  characterize 
the  system.  The  theory  for  doing  so  will  be  briefly  outlined 
in  Section  2.0. 

As  indicated,  the  assignment  of  value  to  a multiple 
attributed  option  is  dependent  on  an  adequate  specification 
of  the  situation  in  which  that  option  will  be  deployed,  and 
one  approach  to  representing  that  future  is  the  use  of 
scenarios.  Section  3.0  briefly  discusses  the  use  of  scenarios 
as  a solution  to  the  so-called  "bushy  mess"  problem  in 
decision  analysis. 

Section  4.0  provides  an  example  of  the  application  of 
value  assessment  procedures  to  evaluating  alternative  WWMCCS 
systems.  Finally,  Section  5.0  further  illustrates  the 
application  of  utility  assessment  procedures  to  the  C^ 
problem  by  using  a specific  problem  characterized  as  follows. 
The  military  can  procure  and  deploy  one  of  N C^  systems. 

The  performance  of  each  of  the  N systems  will  be  observed 
(by  simulation,  war  gaming,  or  some  other  strategy)  in  each 
of  M scenarios.  For  each  system  in  each  scenario,  a meaning- 
ful performance  measure  can  be  obtained.  Also,  for  each 
system,  a credible  cost  estimate,  say  life-cycle  cost,  is 
available.  Given  this  information,  which  system  should  the 
military  service  procure?  The  discussion  will  attempt  to 
illustrate  alternative  procedures  for  the  application  of  the 
principles  discussed  in  Sections  2.0,  3.0,  and  4.0. 
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2.0  THE  ASSESSMENT  OF  VALUE  FOR  MULTIPLE 
ATTRIBUTED  ALTERNATIVES:  A BRIEF  OVERVIEW 

Determining  the  "best"  of  several  complex  alternatives 
involves  the  assessment  of  the  value^  associated  with  multiple 
attributed  alternatives  (often  called  multi-attribute  utility 
assessment,  MAUA) . The  methodology  to  be  discussed  results 
from  the  theory  underlying  the  MAUA  approach,  and  a discussion 
of  that  approach  follows. 

An  example  of  the  theoreticax  problem  being  addressed 
is  the  establishment  of  X2,  and  X^  where 

= $C^, 

X2  = $Weapons,  and 
X^  = $Personnel 

subject  to  the  constraint  that  Xj^  + X2  + ^ K.  Further, 

it  is  assumed  that  value  increases  directly  with  each  increase 
of  Xj^,  X2 , and  X^.  Also,  for  any  particular  vector  X,  a 
program  or  option  can  be  identified  that  maximizes  U.S. 
capability  and  thus  maximizes  value  conditional  on  that 
budget  allocation.  (As  indicated,  the  procedures  for  doing 
so  will  be  discussed  in  Section  4.0.)  Thus,  the  specification 
of  a particular  vector,  X = (Xj^,  X2,  X^)  , specifies  a 
particular  combination  of  capabilities  with  respect  to  C^, 
weapons,  and  personnel.  The  approach  to  be  discussed  assumes 
that  decisions  are  made  under  certainty;  in  other  words,  the 
values  of  Xj^,  X2»  and  X^.  This  is,  of  course  a simplification 
for  any  decision  involving  judgments  and  preferences  with 


A distinction  is  often  made  between  "value"  and  "utility, 
the  latter  term  being  used  in  decision-making  involving 
risk.  Since  risk  is  not  discussed  here,  the  word  "value" 
will  be  used  in  this  discussion. 
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respect  to  future  outcomes  necessarily  involves  uncertainty. 

The  theory  and  procedures  for  assessing  values  under  conditions 
of  uncertainty  must  address  the  question  of  risk  preferences. 
This  fact  complicates  the  nature  of  independence  conditions 
necessary  for  different  functional  forms  of  the  value  func- 
tion as  well  as  procedures  for  assessing  the  function. 

However,  since  the  procedures  to  be  discussed  can  be  expanded 
to  incorporate  risk  preferences  when  the  occasion  requires, 
this  discussion  will  be  limited  to  the  first  step,  value 
assessments . 

The  approach  requires  the  assessment  of  a value  function 
U,  defined  on  the  vector  space  x X2  x X^)  , which  maps 

each  combination  of  C^,  weapons,  and  personnel  into  some 
value.  The  problem  is  to  assess  U and,  in  doing  so,  to 
identify  the  values  Of  X^^,  X2/  and  X^  so  that  U is  maximized 
given  a particular  budget  constraint,  X^^  + X2  + X^  £ K. 

As  an  aside,  there  is,  of  course,  the  practical  problem 
of  the  identity  of  the  decision  maker  whose  preferences  U 
represents.  Is  it  the  President  of  the  U.S.?  Is  it  society? 

In  order  to  assess  U,  it  will  be  necessary  to  trade  off 
different  levels  of  X^^,  X2,  and  X^.  Often,  a high-level 
decision  maker  capable  of  making  policy  trade-offs  does  not 
have  a feel  for  the  marginal  benefit  to  be  derived  from 
changes  in  the  levels  of  particular  X.,  and  the  person  able 
to  assess  the  value  to  be  associated  with  levels  of  X^ 
cannot  make  the  policy  trade-offs.  This  problem  is  a prac- 
tical one  in  decision  analysis,  the  identification  of  decision 
makers  who  have  the  information  and  influence  to  make  and 
implement  decisions.  As  such,  part  of  the  topic  is  addressed 
in  the  literature  on  social  utility  functions  and  will  not 
be  pursued  here.  It  will  be  assumed  that  there  exists  a 
decision  maker  who  can  accurately  express  preferences  among 
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alternative  vectors  in  the  space  and  that  these  preferences 
reflect  some  definable  level  of  policy  or  doctrine. 

2 . 1 The  Ordinal  Approach 

The  ordinal  approach  to  problem  solution  is  based  on  the  - 
assumption  that  U is  an  ordinal  preference  function  such  that 

U maps  each  vector  X = (X^^,  X^)  into  the 

real  numbers  and 

U increases  directly  with  each  X^. 

The  problem  of  assessing  U and  identifying  the  optimal  (X^, 

X^)  can  be  viewed  as  a non-linear  programming  problem  where  the 

objective  function  U is  non-linear  and  the  constraint  ? X,  < K 

1 -L  — 

is  linear. 

As  an  illustration  of  the  approach,  consider  the  solution 
to  the  question  of  the  optimal  levels  of  only  the  variables  X^^ 
and  ^2  ^he  present  assumption  that  X^  can  be  held  constant  in 
the  analysis.  The  constraint  is  that  X^  + X2  £ P ^ K. 

Since  U is  assumed  to  reflect  an  ordinal  preference  func- 
tion, U (X)  is  unique  up  to  any  increasing  transformation 
F(U(X))  of  U.  The  task  is  to  assess  marginal  trade-offs 
between  different  levels  of  X^^  and  X2  in  order  to  ascertain  the 
ordering  imposed  on  the  space  X by  the  function  U. 

In  Figure  2-1,  a set  of  iso-preference  contours  for  Xj^  and 
X2  is  displayed.  The  decision  maker  has  expressed  indifference 
among  all  points  that  comprise  a particular  curve,  with  values 
increasing  with  movement  of  curves  away  from  the  origin. 
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Figure  2-1 

A SET  OF  ISO-PREFERENCE  CONTOURS  FOR  AND  Xj 
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SWEAPONS 


Figure  2 2 

ILLUSTRATtON  OF  PROBLEM  SOLUTION 
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The  point  X*  has  the  simple  property  that  any  increase  in 
either  or  X^,  which  is  necessarily  matched  by  a corresponding 
decrease  in  the  other  dimension,  will  result  in  a lower  overall 
value  than  is  associated  with  the  point  X*. 

Since  U(X*)  is  a maximum, 


3U  + 3U  dX2  _ n r V* 
3X,  3X2dxi  ° - ' 


and  since  ^2  ~ 


1 + dX2  = 0 and 


GA2 


dX2  = -1 . 
dX]^ 


1. 


Substituting  in  equation  1.  above: 


3U 

~ TX2 


0 , or 


9U 

Tx^ 

Tu“ 

7x. 


1 at  X*  . 


2. 


Equation  2 implies  that  at  the  point  X*,  the  decision  maker 

2 

will  give  up  one  dollar  in  C only  if  compensated  by  one 
dollar  in  weapons  and  vice-versa. 

Consider  the  set  of  iso-preference  contours  displayed 
in  Figure  2-3.  Here,  the  "flatness"  of  the  indifference 
curves  implies  that  the  decision  maker  will  give  up  a lot  of 
Xj^  to  obtain  a small  increase  in  X2.  As  illustrated  in  this 
case,  the  point  X*  occurs  for  X^^  higher  than  X2,  but  it  is 
still  the  case  that  equation  2 holds.  The  next  question 
involves  the  process  of  finding  the  point  X*. 
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p 


Figure  2-3 

ISO  PREFERENCE  CONTOURS  AND  X* 
WHEN  X2  IS  MORE  IMPORTANT  THAN  Xi 


13 


4 


A major  difficulty  with  establishing  the  necessary  U 
function  involves  the  number  of  options  (X)  to  be  evaluated. 
This  number  is  potentially  infinitely  large,  and  this  is 
especially  true  if  more  than  just  three  dimensions,  such  as 
those  used  in  the  simplified  example,  are  involved.  Multiple 
trade-offs  are  difficult  to  assess  if  more  than  two  dimensions 
are  involved,  and,  thus,  if  the  dimensions  are  defined  so 
that  complex  value-wise  dependencies  exist  between  them,  the 
assessment  of  U can  be  prohibitively  difficult. 


i 

I 


However,  it  is  often  the  case  that  the  dimensions  can 
be  defined  so  that  a condition  known  as  deterministic 
additive  independence  (DAI)  or  equivalently,  mutual  preference 
independence,  holds  (see  Keelin,  1976  for  discussion  of  DAI 
and  Keeney  and  Raiffa,  1976  for  a discussion  of  mutual 
preference  independence).  Specifically,  this  condition  holds 
for  three  dimensions  (X.,  X.,  and  X.)  if  trade-offs  between 

1 J K 

levels  of  X.  and  X.  for  a fixed  level  of  X,  do  not  depend  on 

the  level  at  which  X,  is  fixed.  If  dimensions  can  be  defined 

k 

so  that  this  condition  holds  for  all  triples  of  dimensions, 
then  the  nature  of  U can  be  established  by  successive  assessments 
of  trade-offs  between  pairs  of  dimensions.  This  process 
involves  careful  definition  of  dimensions  and  then  checking 
for  DAI  by  assessing  indifference  functions  between  levels 
of  X^  and  X^ , such  as  those  displayed  for  X^  and  X^  in 
Figures  2-1  and  2-2,  with  the  level  of  Xj^  fixed.  DAI  implies 
that  the  iso-preference  contours  in  the  (X^,  X^)  plane  will 
remain  invariant  as  a function  of  different  levels  of  the 
fixed  Xj^. 


Assume  that  dimensions  have  been  defined  so  that  DAI 
holds . 


8U 


Define  ~ 


i j 9U 
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where  X^.  represents  the  marginal  rate  of  substitution  between 

levels  of  X.  and  X.. 

1 1 

which  U is  conditionally  maximized,  will  be  1.0, 


Recall  that  Xj^2  evaluated  at  X*,  for 


Further  define  Z^j(X)  = 


3X  . . 
1] 

3X  . 

1 


4 . 


X . . 
11 


where  Z..(X),  known  as  the  marainal  value  reduction  coefficient, 

i;)  — 

reflects  the  manner  in  which  the  marginal  rate  of  substitution 
between  X^  and  X^  changes  as  a function  of  X^.  (See  Keelin  , 

1976 , for  further  discussion  of  the  marginal  value  reduction 
coefficient.)  Note  that  both  X^^  and  (X)  can  be  assessed 
by  using  appropriate  assessment  procedures.  The  questions 
asked  involve  changes  in  preferences  with  differing  levels 
of  X^  and  X j . Consider  the  situation  in  Figure  4. 


Figure  2-4 

CHANGING  ISO  PREFERENCE  CONTOURS  AS  A FUNCTION  OF  X-, 

For  curve  B,  the  decision  maker  will  give  up  more  X^^  for 
specific  amounts  of  X^  than  he  will  for  curve  A.  It  is 
quite  reasonable  to  assume  that  as  one  accumulates  more  Xj^, 
the  incremental  value  of  more  Xj^  as  opposed  to  more  Xj  will 
decrease;  curves  A and  B display  such  a preference  pattern. 
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Keelin  (1976)  proves  that  if  an  additive  form  of  U 
exists,  that  is,  if 

U(X)  = ru^ (X^) , 5. 

i 


then  depends  only  on  X^  and  will  be  the  same  for  all  j. 

Thus,  for  this  case,  Z..(X)  can  be  simply  denoted  as  Z.  and 

11  — 1 

need  only  be  assessed  for  each  i.  Z^  can  be  approximated  as 
f ol lows : 


Z . 


1 


+ AX  . ) - 
1 


6. 


and  Keelin  has  further  shown  that  U(X)  can  be  written  in  an 
additive  form  if  and  only  if 


U{X) 


-^Zi(Xi) 


dx. 


7. 


This  implies  that  in  the  case  when  an  additive  form  of  U 
exists,  determining  the  set  {Z^}  for  all  i completely  specifies 

U. 


Some  examples  of  different  forms  of  an  additive  U are 
the  following: 


= 0 for  X^  implies  U(X)  = a^x^  + ZUj(X^), 


Z^  = 0 for  all  X.  implies  U(X)  = raj^Xj^, 

i 


^i  ~ ^i'  ^ constant,  for  X^  implies  U(X)  = 


10 


->iXi  ♦ EU,(X  ), 

1 j!<i 


•I.  — — 
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1+3  for  X.  implies 

X.  ^ 

1 

U(X)  = + lUj(Xj) 


1’ 


Thus,  by  assessing  appropriate  marginal  trade-offs,  the 

nature  of  U can  be  determined.  Note  that  with  appropriate 

independence  conditions  holding,  the  decision  maker  need  only 

answer  questions  of  the  kind,  "Given  X.  and  X.  at  levels  X.° 
o ' 1 : 1 

and  Xj  , what  change  in  X^  would  just  compensate  for  a decrease 

of  AX^  in  X^?"  By  using  these  questions,  the  ordinal  approach 

can  be  used  to  ascertain  the  nature  of  the  U function  and  to 

rank  order  the  actual  options  or  even  potential  options  under 

consideration. 


Also,  once  the  rank  order  has  been  established,  the 

nature  of  trade-offs  between  dimensions  can  be  given  meaning 

by  trading  off  all  options  into  one  dimension.  For  example,  to 

2 

give  meaning  to  the  trade-offs  between  X^^  = $C  , X2  = $Weapons, 

and  X^  = SPersonnel,  any  option  X = (X^,  X2,  X^)  can  be  compared 

to  options  of  the  form  (X°,  X^i  X°)  where  X°  = $C^  fixed  at  the 

status  quo  and  X°  = $Personnel  fixed  at  the  status  quo.  A 

value  of  X2(  is  found  such  that  X is  indifferent  to  (X°, 

X2,  X°) . In  this  way,  the  different  options  can  be  scaled  in 

terms  of  equivalent  benefits  to  be  derived  with  respect  to  the 

weapons  dimension  by  assuming  some  fixed  levels  of  X^  and  X^.  The 

decision  maker  in  effect  knows  that  a particular  vector  X is 

2 

indifferent  to  some  allocation  of  $Weapons  given  $C  and 
SPersonnel  fixed  at  some  nominal  levels. 

It  should  be  cautioned  that  the  trading  off  of  all  attri- 
butes into  one  must  be  done  with  a judicious  choice  of  fixed 
nominal  levels  of  all  other  attributes.  The  fixed  levels  must 
be  such  that  the  judgments  required  of  the  decision  maker  make 
sense,  and  that  there  actually  does  exist  an  X,  that  satisfies 
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the  equation 


U(X^,  X^,  X^)  = U(X°,  X',  X°).  12. 

Often,  one  of  the  dimensions  over  which  U (X)  can  be 
assessed  is  dollars.  This  is  especially  true  when  dollars  are 
an  attribute  with  respect  to  which  the  decision  maker  is  accus- 
tomed to  making  trade-offs.  In  private  industry,  for  example, 
short-term  profit  can  be  traded  off  against  research  and  design 
programs.  Thinking  in  terms  of  dollar  trade-offs  is  likely  to 
be  difficult  for  a military  commander  in  a particular  opera- 
tion, for  the  attribute  dollars  is  at  that  moment  only  of 
marginal,  if  any,  importance.  It  is  possible  that  at  some 
higher  level  of  the  decision-making  hierarchy,  dollars  are 
relevant.  However,  in  government  decisions  involving  cash 

flows,  trade-offs  will  be  of  the  kind  discussed  here,  that 
2 

is,  between  $C  and  $Weapons  or  between  SMilitary  and  SSocial 
Welfare.  In  other  decisions,  direct  dollar  trade-offs,  such 
as  dollars  versus  energy  consumed,  dollars  versus  lives  lost, 
and  the  like,  may  be  possible. 

2 

Note  that  a particular  level  of  $C  or  $Weapons  provides 
a specific  "best"  program,  but  even  the  best  program  may  not 
be  worth  the  dollars  expended.  Thus  the  trade-off  here  can  be 
between  the  benefits  associated  with  particular  levels  of  each 
attribute  and  cost  in  terms  of  dollars.  What  is  this  program 
really  worth  to  a decision  maker? 

Suppose,  for  simplicity,  that  instead  of  X^  = SPersonnel, 

X^  = simply  $.  X^  corresponds  to  the  status  quo  expenditure 

on  weapons,  and  X?  corresponds  to  the  status  quo  expenditure 
2 

on  C . Then,  as  discussed,  trade-offs  like  those  in  Figure 
2-5  can  be  made. 
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X,  -c2 


Figure  2-5 

TRADE  OFFS  BETWEEN  $ AND  C^,  AND  $ AND  WEAPONS 


2 , 2 The  Interval  Scale  Approach 

The  second  approach  to  establishing  the  optimal  levels  of 
X^,  X^,  and  assumes  that  U is  an  interval  scale  value 
function  unique  up  to  a linear  transformation.  This  assumption 
implies  that  the  decision  maker,  in  assessing  trade-offs,  is 
providing  metric  information  about  differences  in  utilities. 

Consider  the  following  example  involving  the  four 
indifference  curves  displayed  in  Figure  2-6. 
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Figure  2-6 

FOUR  ISO  PREFERENCE  CONTOURS  IN  x X2  SPACE 

The  ordinal  approach  described  in  Section  2.1  implies 
that  the  points  (X^,  X2)  and  (X^®,  X2^  ) in  Figure  2-6  are  of 
equal  utility.  Similarly,  U(X^^,  X2'^)  > U(Xj^^,  X2^)  . No  more 
information  is  provided  in  these  judgirents.  The  interval 
scale  approach  can  provide  information  of  the  kind 


In  fact,  the  utility  differencen  are  assumed  to  be  scaled 
on  a ratio  scale  unique  up  to  a ratio  transformation.  Such 
metric  judgments  are  necessarily  more  difficult  than  ordinal 
preferences . 


However,  there  is  considerable  evidence  that  decision  makers 

2 

can  validly  provide  such  assessments.  The  procedure  for  using 
this  approach  is  similar  to  that  for  using  the  ordinal  approach. 
Assumptions  about  DAI  are  checked  and  pairwise  trade-offs 
between  dimensions  are  assessed.  However,  metric  information 
concerning  the  relative  magnitudes  of  value  difference  is 
derived  from  judgments  by  using  appropriate  procedures.  Often 
it  is  the  case  that  U is  an  additive  value  function  of  the 
form  in  equation  14. 

L’(X)  = a^Uj_(X^)  + aj_U2(X2)  + a3U2(X3)  + a^ . 14. 

U is  an  interval  scale  value  function  as  are  the  individual 
Ui(Xi)  functions.  In  order  for  this  form  of  U to  hold,  the 
independence  assumptions  of  Section  2.1  must  hold,  and  it  also 
must  be  the  case  that  the  DAI  condition  holds  not  only  in  the 
ordinal  form,  but  in  the  interval  form;  that  is,  the  metric 
information  provided  by  trade-offs  between  dimensions  is  inde- 
pendent of  levels  at  which  other  dimensions  are  fixed. 

The  interval  scale  approach,  when  appropriate,  can  be  used 
to  relate  changes  in  benefits  with  respect  to  specific  dimen- 
sions to  cost  and  thus  allow  cost-benefit  analyses  to  proceed 
not  only  at  a global  option  level  (considering  simultaneously 
all  three  dimensions  Xj^,  X2,  and  X^)  > but  also  at  specific 

dimensional  levels.  For  example,  a statement  similar  to  the 

2 

following  can  be  made.  "Increased  C capability  costing  100 
million  dollars  more  than  today's  capability  is  worth  twice 
as  much  as  an  increase  in  weapons  capability  costing  75  million 
dollars."  Such  trade-offs,  possible  also  in  a modified  form 
by  using  the  ordinal  approach,  are  quite  desirable  for  answering 
the  question  of  trade-offs  among  budget  categories. 


2 

W.  Edwards  and  A.  Tversky,  eds.,  "Introduction,"  Decision 
Making  (Middlesex;  Penguin  Books,  1967),  pp.  7-10. 
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An  example  of  just  how  the  interval  scale  procedure  can  be 
used  is  provided  in  Section  4.0.  Specifically,  the  procedure 
is  applied  to  the  problem  of  examining  World  Wide  Military 
Command  and  Control  System  (WWMCCS)  capabilities  and  funding 
levels . 


3.0  STRUCTURING  THE  PROBLEM:  THE  USE  OF  SCENARIOS 


In  Section  2.0,  the  theory  of  the  MAUA  approach  was 

introduced.  Of  major  importance  in  that  approach  is  the 

necessity  for  subjective  judgments  about  trade-offs  among 
2 

attributes.  A C system  can  be  represented  as  a multi- 
attributed  alternative  having  capabilities  on  a large  number 
of  dimensions  or  attributes.  The  relative  importances  of 
capabilities  with  respect  to  different  attributes  can  be 
represented  by  inter-attribute  trade-offs.  However,  as 
discussed  in  the  introduction  to  this  report,  such  trade- 
offs are  conditioned  upon  the  values  of  a large  number  of 
conditioning  variables,  acts  and  events  that  determine  the 
specific  set  of  circumstances  in  which  the  system  will  be 
deployed.  As  indicated,  accurately  representing  the  uncer- 
tain future  in  which  the  systems  are  to  be  deployed  corres- 
ponds to  the  so-called  "bushy  mess"  problem  in  decision 
analysis. 

The  utility  to  be  attached  to  a system  characterized  by 
a set  of  parameters  that  are  related  to  the  eventual  perfor- 
mance of  the  system  can  only  be  determined  when  the  outcome 
of  deployment  is  known.  Obviously,  that  outcome  will  occur 
too  late  to  be  of  any  help  in  system  choice,  and,  therefore, 
an  estimate  of  that  utility,  an  expected  utility,  must  be 
assessed.  The  typical  procedure  for  doing  so  would  involve 
creation  of  a decision  tree  containing  all  relevant  acts  and 
events  interrelated  in  an  appropriate  fashion. 

The  creation  of  such  a decision  tree  begins  small,  by 
considering  only  the  most  critical  acts  and  events.  Other 
acts  and  events  are  added  as  it  becomes  evident  that  the 
decision  under  consideration  could  change  as  a function  of 
their  importance.  In  the  system  design  or  choice  problem. 
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the  number  of  acts  and  events  quickly  becomes  prohibitive, 
and  the  decision  tree  becomes  a bushy  mess  which  must  be 
pruned.  Pruning  essentially  involves  appropriately  approxi- 
mating the  decision  tree. 

Other  modeling  approaches  avoid  the  bushy  mess  problem 
by  characterizing  the  problem  in  such  a way  that  non-critical 
aspects  are  eliminated  as  modeling  proceeds.  Such  elimina- 
tion, however,  must  be  based  on  accurate  representation  of 
the  uncertainty  inherent  in  the  problem  and  on  the  identi- 
fication of  variables  that  discriminate  between  decisions. 
Such  an  analysis  necessarily  deals  in  a different  fashion 
with  the  same  problems  to  be  discussed. 

The  decision  tree  approach  is  one  way  to  address  the 
problem  of  characterizing  the  uncertain  future  in  which  a 
system  will  be  deployed.  Alternative  approaches  are  avail- 
able as  exemplified  by  the  voluminous  literature  on  fore- 
casting the  future  for  different  purposes.  O'Connor  and 

Edwards  have  outlined  the  general  problem  discussed  here  and 

2 

have  suggested  potential  solutions.  They  approach  the 
problem  by  considering  the  use  of  scenarios  in  system  evalua- 
tion. The  problem  addressed  is  identical  to  that  discussed 
thus  far.  In  order  to  assess  the  utility  of  a system  to  be 
deployed  in  an  uncertain  future,  that  future  must  somehow  be 
adequately  represented.  One  way  to  do  so  is  by  the  use  of 
scenarios . 

A scenario  can  be  equated  with  a branch  of  a decision 
tree;  that  is,  a scenario  can  be  viewed  as  a sequence  of 
events  that  sets  the  stage  for  the  deployment  of  a C system. 


Raiffa,  Decision  Analysis:  Introductory  Lectures  on 

Choices  Under  Uncertainty  (Reading,  MA:  Addison-Wesley , 

1968)  . 

2 

.M.  F.  O'Connor  and  W.  Edwards,  On  Using  Scenarios  in  the 
Evaluation  of  Complex  Alternatives,  Technical  Report  TR  76-17 
(McLean,  VA : Decisions  and  Designs,  Inc.,  December  1976). 
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Equivalently,  a scenario  can  be  viewed  as  the  specification 
of  the  values  of  a number  of  relevant  (conditioning)  variables 
upon  which  the  outcome  of  and  utility  associated  with  system 
deploimient  are  dependent.  The  specificity  of  a scenario  can 
be  characterized  by  the  number  of  conditioning  variables 
specified  or,  alternatively,  by  the  length  of  the  branch  of 
the  decision  tree  represented  by  a scenario. 

The  branches  of  a decision  tree  can,  therefore,  be 

viewed  as  a group  of  scenarios.  Similarly,  the  vector  space 

that  is  created  by  taking  the  cross  products  of  all  variables 

upon  which  the  utility  of  system  deployment  could  be  dependent 

can  be  characterized  as  a scenario  space.  In  either  case, 

as  already  indicated,  the  number  of  potentially  relevant 

scenarios  is  prohibitively  large.  Not  all  of  them  can  be 

2 

analyzed.  Yet,  in  the  deployment  of  a particular  C system, 
(or  other  weapons  system)  the  failure  to  capture  the  essence 
of  the  future  in  which  the  system  will  be  deployed  can  and 
probably  will  lead  to  a system  that  is  far  from  optimal, 
perhaps  far  from  satisfactory. 

It  is  essential  to  establish  criteria  for  choosing 
scenarios.  Two  criteria  that  are  reasonable  but  may  not  be 
com.patible  have  been  labeled  as  "representativeness"  and 
" discriminability . " ^ 

The  representativeness  criterion  refers  to  the  fact 
that  in  an  evaluation  system  deployment,  it  is  necessary  to 
represent  accurately  the  future  in  which  the  system  will  be 
deployed.  Although  not  every  specific  event  can  be  captured 
by  a representation  of  the  future,  no  critical  event  or 
class  of  events  can  be  ignored.  In  representing  a popu- 


^Ibid. 


lation  of  voters  in  an  opinion  poll,  no  critical  sub-stratum 

of  the  population  can  be  ignored  or  misrepresented  without 

danger  of  a significant  error  of  estimation.  Similarly,  in 

representing  the  future  for  system  deployment,  no  critical 

set  of  conditioning  variables  or  critical  region  of  the 

decision  tree  can  be  ignored  or  misrepresented.  Answering 

2 

the  question  of  when  the  U.S.  has  enough  C capability 
necessarily  involves  some  projection  of  the  future,  and  that 
projection  must  include  representative  scenarios. 

Recall  that  one  problem  addressed  in  this  report  is 

2 

establishing  the  optimal  C system  that  can  be  obtained  for 
$X^.  The  solution  to  this  problem  necessarily  involves 
choosing  among  alternative  proposed  systems.  Yet,  most 
system  designers  try  to  design  systems  to  be  adequate  with 
respect  to  representative  threats  or  representative  futures. 
The  systems  may  differ  somewhat  in  their  ability  to  handle 
certain  aspects  of  the  situation,  but  all  in  all,  the 
difference  among  systems  m.ay  not  be  large.  In  order  to 
choose  among  systems,  it  is  often  necessary  to  create  sce- 
narios that  discriminate  in  a value  sense  among  the  options 
under  evaluation,  and  these  scenarios  may  not  be  at  all 
representative  of  the  future.  These  scenarios  will  guarantee 
a difference  among  systems  in  terms  of  value,  but  how  impor- 
tant is  the  difference? 

Representative  scenarios  are  thus  not  necessarily 
discriminative,  and  a set  of  scenarios  that  satisfies  both 
criteria  can  be  difficult  to  create.  One  solution  is  to  use 
a two-step  procedure.  First,  systems  are  evaluated  in 
scenarios  that  are  representative  of  the  future  and  are 
mutually  exclusive  and  exhaustive.  The  expected  value  of  a 
system  is  found  by  multiplying  the  probability  of  each 
scenario  by  the  value  of  the  system  in  the  scenario  and 
aggregating  these  weighted  values.  If  large  differences  in 
expected  value  exist,  further  evaluation  may  not  be  necessary. 
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If  no  system  is  satisfactory,  the  creation  of  new  options 
may  be  necessary.  If,  however,  a reasonable  number  of 
systems  are  close  and  satisfactory  in  expected  value,  fur- 
ther evaluation  may  be  necessary. 

A method  of  establishinj  discr iminitive  scenarios, 
labeled  "single  threading,"  is  the  following.  The  attributes 
of  scenarios  that  would  make  each  system  look  very  good  are 
identified.  These  attributes  would  be  those  that  enhance  or 
take  advantage  of  the  better  aspects  of  the  system.  Similarly, 
attributes  that  degrade  system  performance  are  also  identified. 
Then,  for  each  proposed  system,  a set  of  scenarios  is  created 
which  em.phasizes  the  positive  aspects  of  the  system  and  de- 
emphasizes  the  negative  aspects  of  the  system. 

Each  system  is  evaluated  in  all  scenarios,  those  in 
which  that  system  should  achieve  excellent  performance  as 
well  as  those  designed  to  enhance  others.  The  system  per- 
formances in  scenarios  must  then  be  aggregated  across  sce- 
narios, but  the  procedure  of  weighting  the  value  of  the 
system  in  each  scenario,  multiplying  by  scenario  probability, 
and  aggregating  is  not  necessarily  appropriate.  One  reason 
is  that  the  scenarios  are  not  necessarily  mutually  exclusive 
and  exhaustive  as  is  the  case  with  a set  of  representative 
scenarios.  Also  because  no  partition  of  the  future  exists, 
establishing  the  actual  scenario  probabilities  may  be  a 
fairly  difficult  absolute  judgment  task. 

A possibility  would  be  zo  establish  the  conditional 
probability  of  each  discriminitive  scenario,  given  that  one 
of  the  representative  scenarios  has  occurred,  and  to  multiply 
that  conditional  probability  by  the  probability  of  the 
representative  scenario.  This  procedure  is  repeated  for 
each  representative  scenario,  and  the  products  are  summed 
over  the  set  of  representative  scenarios  to  obtain  the 
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probability  of  the  discriminative  scenario.  This  proba- 
bility may  be  used  as  a weight  for  the  differences  in  option 
values  in  the  scenarios. 

An  important  point  must  be  made  with  respect  to  option 
v'alues  in  scenarios.  One  view  of  the  value  of  an  option  in 
a scenario  is  the  value  of  the  expected  consequence  of  de- 
ploying that  option  in  the  scenario.  However,  that  consequence 
may  have  little  to  do  with  the  system.  Perhaps  no  system, 
would  do  well  in  the  scenario.  For  example,  in  a strategic 
nuclear  confrontation,  the  consequences  are  likely  to  be 
disastrous  no  m.atter  which  system  is  deployed.  There  is, 
in  effect,  a guaranteed  large  negative  value  associated  with 
certain  low  probability,  high  importance  scenarios,  and 
since  no  system  can  do  much  to  reduce  that  negative  value, 
all  systems  will  have  low  value  in  such  scenarios.  Similarly, 
certain  high  probability  scenarios  like  the  involvement  of 
the  U.S.  Navy  in  normal  operations  in  the  West  Pacific  involve 
fairly  minimal  consequences  and  the  relative  impact  of  all 
systems  is  about  the  same. 

Accordingly,  the  weight  given  a scenario  should  be  a 
function  not  only  of  scenario  probability,  but  also  of  the 
importance  of  the  scenario.  The  im.portance  of  a scenario  is 
a function  of  possible  system  impact  in  the  scenario. 

Scenarios  where  no  system  can  improve  the  situation,  that 
is,  those  in  which  the  value  of  the  consequences  of  deploy- 
ment of  any  system  is  extremely  negative,  should  receive  low 
weight.  And  scenarios  where  the  consequences  of  poor  system 
performance  are  minimal  should  also  receive  little  weight. 

A plausible  procedure  for  weighting  discriminative 
scenarios  is  the  following: 

1.  For  each  scenario,  the  value  of  the  consequences 

of  deployment  of  the  worst  feasible  and  also  the  best 
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feasible  system  are  established  as  minimum  and  maximum 
system  values  for  that  scenario.  The  difference  between 
these  values  is  called  the  value  difference  for  a 
scenar io . 

2.  The  largest  value  difference  across  scenarios  is 
assigned  an  arbitrary  value,  say  100  or  so,  and  value 
differences  for  other  scenarios  are  scaled  by  compari- 
son with  this  difference.  Note  that  in  order  to  make 
such  comparisons,  value  differences  must  be  well  defined. 
An  example  of  a good  definition  would  be  the  dollar 
value  of  equipment  and  lives  lost  as  a consequence  of 
system  deployment  in  the  scenario.  Perhaps  such  a 
precise  definition  is  difficult,  but  such  ratings  do 
require  that  there  exist  a sound  basis  for  comparison 

of  value  differences  across  scenarios. 

3.  The  scaled  value  difference  of  each  scenario  is 
multiplied  by  the  scenario  probability  to  yield  an 
expected  value  difference. 

4.  The  expected  value  differences  are  normalized  to 
sum  to  1.0  across  all  scenarios.  These  normalized 
values  serve  as  scenario  importance  weights. 

This  procedure  requires  that  scenarios  do  not  intersect, 
that  is,  that  the  conditional  probability  of  one  given  the 
other  is  zero.  If  this  is  not  the  case,  value  differences 
may  be  overweighted  by  being  counted  in  two  scenarios.  When 
scenarios  intersect,  they  must  be  separated  by  introducing 
the  negation  of  one  as  a detail  of  the  other  or  by  com.bining 
them  into  one  scenario. 

Once  the  set  of  scenario  importance  weights  has  been 
established  and  checks  have  been  made  for  overweighting,  the 
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value  of  each  system  is  assessed  on  a O-to-100  scale  in  each 
scenario  by  comparing  that  system  performance  with  the  per- 
formances of  the  worst,  best,  and  other  systems  in  the 
scenario.  The  value  of  the  system  in  each  scenario  is 
weighted  by  the  scenario  importance,  and  these  weighted 
values  are  summed  across  scenarios  to  yield  an  overall 
system  value. 

The  approach  to  this  problem  thus  far  discussed  is  to 
characterize  each  system  as  a multi-attributed  alternative 
where  the  perform.ance  of  the  system,  in  a scenario  is  one  of 
the  system  attributes.  This  approach  will  be  further  dis- 
cussed in  Section  5.0  where  the  specific  problem  outlined  in 
Section  1.0  is  addressed  in  detail. 
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4.0  AN  EXAMPLE:  THE  DEVELOPMENT  OF  AN  ANALYTIC  PROCEDURE 

FOR  EXAMINING  THE  RELATION  BETWEEN  WORLDWIDE  MILITARY 
COMMAND  AND  CONTROL  SYSTEMS  (WWMCCS)  PERFORMANCE 
CAPABILITIES  AND  FUNDING  LEVELS 


This  section  of  the  report  describes  the  application  of 
the  value  assessment  methodology  to  a hypothetical  problem 
of  evaluating  alternative  architectural  candidates  for  a 
World  Wide  Military  Command  and  Control  System  (WVMCCS).^ 

The  procedure  used  is  an  example  of  the  interval  scale 
approach.  The  analysis  was  performed  to  determine  the 
relative  value  of  potential  WW>1CCS  crisis  requirements.  In 
addition,  a description  of  a procedure  for  extending  the 
analysis  in  order  to  identify  architectural  candidates  that 
yield  maximum  WWMCCS  capability  given  fixed  budget  con- 
straints was  developed. 

4 . 1 Introduction  to  Requirement  Level  Definitions 

This  study  commenced  with  the  formulation  of  a set  of 
general  LTVMCCS  requirements  statements  drawn  from  (1)  a 
review  of  pertinent  Defense  Department  documents;  (2) 
selected  "post  mortem"  observations  about  past  crises 
managemient  situations;  and  (3)  first-hand  knowledge  from 
form.er  participants  in  U.S.  Command  and  Control  System.s. 

From  these  general  requirements,  specific  requirements  were 
derived  and  defined  in  such  a manner  that  each  one  could  be 
related  to  a specific  crises  management  support  need  and 
could  also  be  understood  by  an  operator  (WWMCCS  User)  and  an 
engineer  (Wlv’MCCS  Designer).  An  example  of  such  a specific 
requirement  might  be  the  description  of  the  need  for  an 
interface  among  crises  battle  staff  and  analyst  personnel  in 
the  National  .Military  Command  Center  (NMCC)  and  those  in  the 
command  posts  of  the  Unified  and  Specified  (U  and  S)  Command 


The  example  is  based  on  work  done  for  the  IBM  Corporation, 
Purchase  Order  No.  571219  in  support  of  the  WWMCCS  Contract 
USAF  Contract  F19628-74-C-0158 . 


Once  a comprehensive  list  of  such  performance  require- 
ments was  developed,  each  requirement  was  refined  to  reflect 
different  levels  of  capability  and  sophistication.  For 
example,  should  the  interface  referred  to  above  include  a 
capability  to  transmit  photographic  materials  within  minutes 
from  one  command  node  to  another?  The  different  levels  of 
capability  were  chosen  to  provide  a clear,  understandable 
distinction  in  operational  terms  of  the  different  contributions 
of  each  to  crises  management.  A complete  description  of 
each  requirement  and  the  levels  of  sophistication  within 
each  are  outlined  in  Appendix  A.  The  numbering  system  used 
is  that  used  in  the  original  study  (requirements  3.2.18  and 
3.2.19  are  deleted). 

These  different  levels  of  performance  capability  are 
generally  relatable  to  timeliness,  geographic  coverage, 
command  echelon,  and  other  characteristics  such  as  voice  and 
text  transmission.  This  method  of  stating  requirements 
facilitates  the  evaluation  of  the  contribution  of  different 
levels  of  capability  to  crises  managem.ent  in  different 
scenarios.  This  evaluation  forms  the  basis  for  the  model 
(described  in  Section  4.2)  which  is  used  to  prioritize  the 
relative  importance  of  specific  requirements  and  to  measure 
the  increase  in  benefits  derivable  from  higher  levels  of 
capability  aunong  requirements. 

4 . 2 Description  of  Methodology  and  Interpretation  of  Values 

The  requirement  levels  described  in  Section  4.1  were 
evaluated  by  using  the  m.ethodology  explained  in  Section  4.3. 

The  first  step  of  the  method  involves  determining  the  relative 
value  of  adding  the  highest  capability  level  of  each  require- 
ment to  the  system,  within  a given  situation.  This  step  is 
then  repeated  for  each  situation.  The  next  step  is  to 
determine  the  relative  priority  of  each  situation  for  the 
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Ws’MCCS  design,  a step  accomplished  by  determining  the  rela- 
tive probability  of  occurrence  of  each  situation,  the 
potential  relative  impact  of  that  situation  on  U.S.  interests, 
and  the  relative  contribution  that  a maximum  WWMCCS  system 
makes  tc  the  management  of  each  situation.  Combining  these 
factors  yields  the  relative  priority  sought.  Next,  by 
com.bining  the  relative  value  of  a requirement  within  each 
situation  with  the  priority  of  that  situation  and  summing 
across  all  situations,  the  value  of  the  requirement  is 
determined . 

Having  these  sum.roary  measures  of  value  of  the  range  of 
capability  for  each  requirement,  the  next  step  is  to  determine 
the  value  of  each  level  of  capability  of  the  requirement. 

This  determination  is  made  by  examining  each  requirement  and 
assessing  the  percentage  of  the  total  requirement  value  that 
is  obtained  by  adding  each  level.  This  percentage  contri- 
bution of  each  level  is  then  multiplied  by  the  requirem.ent 
value  to  arrive  at  a value  for  each  level.  (When  the  per- 
centage contribution  of  each  level  is  dependent  upon  the 
situation,  a weighted  average  percentage  is  used  as  the 
multiplication  factor,  with  the  situation  priorities  as 
weights . ) 

These  level  values,  or  measures  of  expected  benefit,  are 
comparable  in  the  same  way  that  the  requirement  values  are 
comparable.  For  instance,  changing  from  the  current  system 
for  requirement  3.2.4  (Vertical-Horizo.-tal  Analysis  Inter- 
face for  Command  Duty  Personnel)  to  the  level  II  improvement 
(value  of  3.1)  is  half  as  valuable  as  changing  from  the 
current  system  to  level  II  for  requirement  3.2.9  (Intelligence 
Interface)  (value  of  6.2).  Furthermore,  for  requirement 
3.2.4,  changing  from  level  II  to  level  III  (value  of  1.5)  is 
half  as  valuable  as  changing  from  level  I to  level  II  (value 
'of  3.1),  and  changing  from  level  III  to  level  IV  (value  of 
1.6)  is  about  as  valuable  as  changing  from  level  II  to 
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level  III  (value  of  1.5).  Also,  the  value  of  changing  from 
the  current  system  to  level  IV  of  requirement  3.2.4  (value 
of  6.2)  is  just  as  valuable  as  changing  from  the  current 
system  to  level  II  of  requirement  3.2.9  (intelligence  inter- 
face) (value  of  6.2).  (The  value  of  going  to  level  III  of 
requirement  3.2.9,  however,  is  about  40%  greater  than  the 
value  of  going  to  level  IV  of  requirement  3.2.4).  The 
reader  can  ma)ce  other  comparisons  by  consulting  the  results 
in  Appendix  A. 

4 . 3 Value  Assessment  Methodology 

In  order  to  arrive  at  the  requirement  valuation  dis- 
played in  Appendix  A,  a methodology  which  allowed  for  the 
comparison  of  the  various  requirements  on  a meaningful  scale 
was  necessary.  Since  the  requirements  considered  were  not 
reducible  to  natural  units  (such  as  dollars) , much  less  to 
the  same  natural  units,  the  methodology  had  to  treat  relative 
values  in  such  a way  as  to  arrive  at  a single  valuation 
number . 

The  steps  followed  in  this  methodology  are: 

1.  Assign  relative  importance  weights  to  requirements 
within  each  situation, 

2.  Determine  the  relative  priority  of  each  situation, 

3.  Combine  the  factors  from  Steps  1 and  2 above  to 
arrive  at  importance  weights  across  situations,  and 

4.  Assess  the  importance  of  each  level  of  capability 
within  each  requirement. 

Table  4-1  is  an  example  of  the  type  of  input  assessed 
in  Step  1.  Displayed  are  normalized  relative  importance 
weights  for  requirements  in  two  situations.  It  shows,  for 
instance,  that  within  "BLOCK"  (blocltade  situation)  , it  is 
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most  important  to  improve  the  current  system  to  the  top 
level  capability  (levels  are  described  in  Appendix  A)  for 
requirement  3.2.5,  IVWMCCS  Communication  for  Senior  Decision 
Makers.  Furthermore,  it  is  three  times  as  important  to 
increase  the  capability  of  this  requirement  as  it  is  to 
increase  the  capability  of  requirement  3.2.1,  NCA-Heads  of 
State  Interface.  The  reason  for  this  relationship  is  that 
since  the  current  capability  of  requirement  3.2.1  already 
provides  for  comm, unication  with  the  U.S.S.R.,  increasing 
this  capability  is  of  little  value  in  the  blockade  situation. 
Sim.ilarly,  since  it  is  doubtful  that  a transportable 
facility  would  be  used  in  the  blockade  situation,  the  rela- 
tive value  of  having  such  a capability  (requirement  3.2.7) 
is  judged  to  be  very  low  (about  1/12  as  valuable  as  the 
increased  communications  capability  of  3.2.5). 

For  comparison  purposes,  next  consider  "QUAKE"  (earth- 
quake situation) . In  this  situation,  it  is  considered  to  be 
very  important  to  have  the  transportable  facility  (require- 
ment 3.27)  to  provide  communications  with  the  disaster  area, 
and  this  requirement  is  assigned  the  highest  relative 
importance  value.  Also  note  that  in  this  situation,  require- 
ment 3.2.5  (which  is  the  most  important  requirement  for  the 
blockade)  is  less  than  half  as  important  as  3.2.7  for  this 
situation.  The  reason  for  this  reversal  in  importance  is 
that  requirement  3.2.5  is  mainly  a system  to  link  overseas 
personnel  with  the  U.S.  and  with  each  other.  Since  the 
earthquake  situation  is  basically  an  internal  problem,  the 
overseas  capability  is  not  particularly  important. 

Given  requirements  prioritized  within  each  situation, 
it  is  next  necessary  to  establish  the  relative  priorities  of 
the  different  situations.  The  three  factors  most  important 
to  situation  priority  are: 
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1.  The  relative  probability  of  situation  occurrence; 

2.  The  relative  impact  on  U.S.  interests  of  each 
situation;  and 

3.  The  relative  contribution  that  a "maximum  WWMCCS" 
would  make  toward  the  management  of  each  situation. 

Relative  probabilities  are  established  by  comparing  the 
likelihoods  of  each  situation  on  a ratio  scale  and  normalizing 
the  values  to  sum  to  1.0.  The  relative  impacts  on  U.S. 
interests  and  also  the  contribution  of  a "maximum  WWMCCS" 
are  established  by  assigning  that  situation  having  the 
greatest  impact  (or  contribution)  a value  of  100  and  comparing 
all  others  to  that  situation  as  well  as  to  each  other.  The 
final  results  are  ratio  scale  impacts  of  impact  and  con- 
tribution values  which  are  normalized  to  sum  to  100. 

A multiplicative  combination  rule  was  chosen  for  com- 
bining the  three  situation  priority  factors  into  an  overall 
priority.  Such  a combination  rule  incorporates  the  obvious 
dependencies  among  the  factors.  For  example,  if  any  one  of 
the  three  factors--situation,  probability,  relative  impact 
on  U.S.  interests — or  relative  contribution  of  a "maximum 
WWMCCS"  has  a zero  value  for  a situation,  the  priority  of 
the  situation  should  be  zero. 

Table  4-2  is  an  example  of  an  elicited  prioritization 
of  situations.  In  this  figure,  each  component,  relative 
probability,  relative  impact  on  U.S.  interests,  and  relative 
contribution  of  a maximum  WWMCCS,  is  assessed  separately  and 
normalized.  The  bottom  row  of  numbers  is  the  normalized 
products  of  the  components.  (The  last  row  of  numbers  appears 
slightly  inconsistent  with  the  first  three  rows  because,  for 
purposes  of  presentation,  each  row  was  individually  rounded 
off  to  the  nearest  whole  number.  The  rounded  product  is 
based  on  the  use  of  more  accurate  component  numbers. ) 
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There  is  now  enough  information  to  determine  the  value 
of  each  requirement  by  taking  a weighted  average.  That  is, 
the  relative  weight  of  each  requirement  within  each  situa- 
tion is  weighted  by  the  priority  of  the  situation  and  averaged 
across  all  situations.  This  process  yields  the  overall 
value  of  each  requirement.  The  results  are  displayed  in 
Table  4-3.  Here  the  last  column  shows  the  value  of  each 
requirement.  This  nurr±)er  has  an  interpretation  similar  to 
that  for  the  numiber  of  each  colum.n  but  independent  of  situa- 
tion. For  example,  it  is  most  valuable  to  include  the  top 
level  capability  of  requirement  3.2.5.  Adding  this  capa- 
bility gains  9.4%  of  the  total  value  of  a maximum  WWMCCS. 

It  is  more  than  twice  as  valuable  to  add  this  capability  as 
it  is  to  add  the  top  level  of  requirement  3.2.7. 

Tnis  analysis  v;as  done  with  several  experts,  and  con- 
vergence across  experts  was  examined.  That  information  is 
not  reported  here.  Table  4-4  displays  mean  values  for  all 
requirements . 

The  final  step  in  the  analysis  involved  determining  the 
value  of  each  capability  level  within  each  requirement. 

Figure  4-1  shows  an  example  of  this  process  for  requirement 
3.2.3.  Ta->le  4-4  shows  that  the  value  of  increasing  the 
capability  of  requirement  3.2.3  from  the  current  system  to 
its  top  level  is  8.9.  Since  requirement  3.2.3  was  described 
by  fo.;r  capability  levels  (see  Appendix  A),  Level  IV  is 
valued  at  100%  of  8.9  or  8.9.  (Recall  that  the  definition 
of  the  value  scale  is  the  improvement  over  the  current 
capability.  Thus,  the  value  for  Level  I,  the  current 
system.,  is  0.)  In  this  case,  a consensus  of  the  assessors 
valued  Level  III  of  the  requirement  at  90%  of  the  total 
value  (8.0)  and  Level  II  at  50%  of  the  total  value  (4.4). 
(Applying  the  same  methodology  to  the  other  requirements 
yielded  the  values  shown  in  Appendix  A.) 


“'.otf  All  numbers  »rr  rounded  to  nejrf^T  integer 
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An  interesting  point  that  arose  in  this  assessment  is 
that,  for  most  requirements,  the  percentage  of  total  value 
gained  at  each  capability  level  is  independent  of  situation. 
This  feature,  while  not  necessary  for  the  methodology, 
greatly  reduces  both  the  elicitation  and  computational 
burden  involved.  (For  requirement  3.2.14,  Crises  Monitoring 
Support,  the  percentage  was  assessed  to  be  different  de- 
pending upon  the  time  urgency  of  the  situation.  In  this 
case,  situations  were  grouped  into  "time-urgent  situations" 
and  "others"  and  a weighted  average  percentage  was  used. 

The  weights  for  t.his  weighted  average  were  the  priority  of 
the  "time-urgent  situations"  and  the  priority  of  the  "other 
situations . " ) 

4 . 4 Some  Implications  of  the  Analysis 

Of  the  15  requxrements  described  and  analyzed  in  this 
study,  and  based  upon  the  crises  scenarios  which  were  given, 
the  value  of  an  improved  system  for  interfacing  senior  U.S. 
decision  makers  (3.2.5)  carries  the  highest  relative  priority 
weighting.  Also  high  among  the  fifteen  is  the  requirement 
for  an  improved  crises  monitoring  capability  responsive  to 
crises  support  for  such  high-level  personnel  as  the  Chairmian 
of  the  Joint  Chiefs  of  Staff,  the  Secretary  of  Defense,  and 
the  NCA  (3.2.14). 

The  analysis  reflects  the  fact  that  the  most  demanding 
crises  scenarios  involved  timie-urgent  situations  requiring 
constant  close  coordination  among  officials  in  the  National 
.Military  Command  Center  (N.MCC)  , the  military  services,  the 
overseas  commanders,  and  established  allies.  Results  gener- 
ally reflect  the  need  to  transmit  visual  materials  such  as 
photography  for  targeting  purposes  and  positive  threat 
identification;  for  secure,  rapid  voice  conferring  among  the 
highest  level  personnel;  and  for  keeping  all  military  elements 
continuously  informed  of  the  developing  situation. 
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The  analysis  also  points  up  the  value  of  a user-oriented 
query  and  display  capability  for  senior  decision  makers  as  a 
part  of  WisMCCS.  There  is  an  obvious  need  to  improve  the 
current  capability  for  updating  personnel  who  monitor  crises 
developments.  The  current  system  is  constrained  largely  to 
secure  telephone  and  personal  exchanges,  the  first  of  which 
is  difficult  and  time-consuming  for  highest  officials  and 
the  second  of  which  has  the  additional  disadv'antage  of  being 
exceedingly  slow. 

Certain  other  requiremients  were  analyzed  as  being  of 
relatively  low  priority.  One,  a transportable  (3.2.7) 
facility,  reflects  the  fact  that  the  current  capability 
inherent  in  command  and  control  type  aircraft,  surface  ships 
and  submarines  is  generally  satisfactory,  at  least  for  the 
scenarios  used.  Another  requiremient  of  relatively  low  value 
includes  a highly  automated  system  for  searching  files, 
alerting  personnel,  preparing  messages,  initializing  analysis 
models,  and  so  on  (3.2.15).  The  low  value,  in  this  instance, 
is  partly  a reflection  of  the  crises  scenarios  which,  for 
the  most  part,  do  not  develop  rapidly  from  a complete  sur- 
prise situation.  Two  situations  are  exceptions;  but  they 
are  also  unique.  The  preprogramming  required  for  a highly 
automiated  system  to  be  responsive  would  not  be  practical. 

Plotting  the  value  of  moving  from  the  current  system,  to 
the  second  level  of  each  requirement  produces  Table  4-5. 

This  diagram  shows  that  it  is  roughly  twice  as  valuable  to 
improve  to  the  second  level  in  requirements  3.2.9,  3.2.11, 
3.2.14,  3.2.2,  3.2.5,  3.2.10,  12,  13,  and  3.2.3  as  it  is  to 
move  to  the  second  level  in  the  other  requirements. 

There  are  two  main  reasons  for  this  difference  among 
requirem.ents . The  most  obvious  reason  is  that,  as  shown  in 
Table  4-4,  some  requirements  are  more  important  than  others. 
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This,  for  example,  is  the  reason  that  level  II  of  require- 
ment 3,2.2,  NCA-WWMCCS  Interface  (value  of  5),  is  much  more 
important  than  level  II  of  requirement  3,2.19,  Intelligence/ 
Operations/Diplomatic  Coordination  (value  of  2.8). 

There  is,  however,  another  reason  for  the  results  shown 
in  Table  4-5  which  causes  this  figure  to  differ  from.  Table  4-4. 
This  reason  is  that,  for  some  requirements,  a very  large 
percentage  of  the  total  value  of  the  requirement  is  obtained 
in  moving  from  the  current  system  to  the  second  level  of  the 
requirement.  Such  is  the  case  with  requirement  3.2.11,  User 
Oriented  ADP-based  Support.  For  this  requirement,  80%  of 
the  total  value  is  obtained  by  adding  the  second  level, 
yielding  a second  level  value  of  6.1.  Thus,  the  second 
level  of  3.2.11  is  more  valuable  than  the  second  level  of 
3.2.5,  WWMCCS  Communications  for  Senior  Decision  Makers 
(where  only  45%  of  the  total  value  is  obtained  by  going  to 
level  II  and  yields  a level  II  value  of  4.9)  even  though  the 
total  value  for  3.2.5  is  much  higher  (10.8  for  3.2.5  compared 
with  7.6  for  3.2.11) . 

4 . 5 Design  Optimization  and  Evaluation  Methodology 

The  total  set  of  WWMCCS  requirements  consists  of : 

o Crisis  Requirements, 

o Theater  Nuclear  Requirements, 

o CONUS  Nuclear  Requirements,  and 

o Day-to-Day  Requirements 

This  discussion  has  dealt  only  with  requirements  in  the 
crisis  area,  but  it  is  a straightforward  extension  of  the 
methodology  to  consider  other  areas  as  well.  In  this  section, 
the  example  analysis  will  be  further  restricted  to  consideration 
of  four  of  the  19  crisis  requirements.  The  analysis  will  be 
extended  to  demonstrate  how  the  cost  factor  can  be  incorporated 
and  to  examine  the  relation  between  system  performance  capa- 
bilities and  funding  levels. 


The  following  four  requirements: 


o NCA  - Heads  of  State  Interface, 
o NCA  - WWMCCS  Interface, 

o Washington  Level  Interagency  Interface,  and 

o WWMCCS  Communications  for  Senior  Decision  Makers 

will  be  considered,  for  the  purpose  of  demonstration,  to  be 
the  total  set  of  requirements.  Table  4-6  shows  these  require- 
ments broken  out  into  levels  as  earlier  described.  The 
"VALUE"  columin  was  determined  by  the  methodology  explained. 

The  cost  figures  are  purely  hypothetical  values  assigned 
strictly  for  demonstrating  the  methodology.  These  numbers 
are  interpretable  as  summary  measures  of  installation  cost 
plus  five-year  operating  costs.  Thus,  the  figures  shown  for 
the  current  system  capability  (level  I in  each  case)  include 
operating  costs  and  are  not  zero.  The  summation  figures  at 
the  bottom  of  the  diagram  show  that  it  would  cost  $163 
million  to  add  the  top  level  of  each  requirement  and  that 
adding  this  capability  would  produce  35%  of  the  benefit 
obtainable  by  adding  the  top  level  of  all  19  requirements. 

In  order  to  perform  an  optimization,  it  is  helpful  to 
consider  not  only  the  cost  and  benefit  of  each  capability, 
but  also  the  incremental  cost  and  benefit  of  adding  a capa- 
bility level.  These  incremental  costs  and  benefits  are 
displayed  in  the  columns  headed  "COST"  and  "VALUE."  These 
columns  show,  for  instance,  that  it  costs  $22  million  to  add 
level  III  of  requirement  1 to  a system  that  already  contains 
level  II  of  this  requirement  and  that,  by  such  an  addition, 
the  system  value  is  increased  by  1.1.  The  final  column 
shows  the  ratio  of  increased  value  to  increased  cost  for 
each  capability  level  increment.  For  instance,  considering 
the  addition  of  level  III  of  requirement  1,  the  benefit  to 
cost  ratio  is  1.1/22,  or  .05.  The  numbers  in  this  column 
show  that  the  most  benefit  per  unit  of  cost  is  obtained  by 
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additions  of  requirement  2 capabilities.  Thus,  in  designing 
a system,  the  first  amount  of  resources  available  should  be 
devoted  to  increasing  the  capability  of  requirement  2. 

Table  4-7  shows  how  this  optimization  is  done.  First, 
consider  designing  the  system  that  provides  the  most  benefit 
within  a $20  mdllion  budget.  This  figure  shows  that  the 
first  $6.1  m.illion  is  devoted  to  operating  at  the  current 
level  of  capability  for  all  requirements.  The  next  $1.7 
m.illion  is  then  devoted  to  raise  the  capability  of  requirement 
2 to  its  highest  level.  Finally,  the  next  $10.2  million  is 
used  to  raise  the  capability  of  requirement  3 to  level  III. 

A total  of  $18  million  is  now  committed,  and  no  further 
improvements  are  possible  (all  other  improvements  cost  at 
least  $7  million).  The  value  of  this  system  configuration 
is  16.4  (of  a maximum  of  35),  and  no  alternative  allocation 
of  the  $20  million  produces  a higher  value.  Similarly,  a 
$110  million  budget  provides  a value  of  29.3  by  including 
the  level  II  capability  of  equirement  1,  level  III  of 
requirement  2,  level  IV  of  requirement  3,  and  level  IV  of 
requirement  4.  Again,  while  there  are  alternative  ways  of 
allocating  the  $110  million,  none  of  these  alternatives 
produce  a higher  value. 

This  example  shows  how  it  is  possible  to  use  this 
methodology  to  optimize  the  WWMCCS  design  within  a fixed 
budget.  In  the  case  of  a full-scale  problem,  the  next  step 
would  be  to  perform  a sensitivity  analysis  on  both  the  cost 
and  value  numbers  to  identify  areas  that  deserve  further 
attention.  For  instance,  consider  the  cost  figures  for 
requirement  2.  Even  if  these  cost  estimates  are  less  than 
one-half  of  the  actual  costs,  the  output  of  the  analysis 
remains  unchanged.  That  is,  level  III  of  this  requirement 
would  be  a part  of  any  budget  over  $20  million.  Thus,  it 
would  be  a waste  of  resources  to  devote  any  effort  to  deter- 
mining better  cost  figures  for  this  requirement  (the  same 
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thing  holds  with  regard  to  further  value  analysis  for  this 
requirement  as  well).  On  the  other  hand,  a slight  increase 
in  the  value  of  requirement  1 or  decrease  in  value  of  re- 
quirement 4 would  cause  the  optimum,  system  for  the  $110 
million  budget  to  include  a higher  capability  level  of 
requirement  1 and  a lower  capability  of  requirement  4,  This 
indicates  that  additional  effort  is  justified  to  refine  the 
estir.ates  for  these  requirements. 

Whether  or  not  a sensitivity  analysis  is  performed  and 
the  values  modified,  the  result  of  the  analysis  provides  a 
useful  guide  to  the  overall  WWMCCS  design.  This  is  demon- 
strated by  Figure  4-2.  This  figure  shows  that  about  one- 
half  of  the  total  value  of  a maximum  WWICCS  can  be  obtained 
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for  about  11%  of  the  cost  of  the  maxin’.um  systerri.  Another 
35%  of  the  total  value  can  be  obtained  for  another  50%  of 
the  total  cost  (of  course,  a refined  model  will  produce 
nuTTibers  that  are  much  more  defensible)  . Such  a result  is 
useful  in  the  determ.ination  of  the  value  of  an  increased 
budget  and  the  determiination  of  a m.inimum  budget. 

4 . 6 Remarks  on  the  Example 

The  major  effort  so  far  has  been  in  the  quantification 
of  crisis  requirements;  for  exam.ple,  the  question  is  not 
whether  messages  are  transmitted  from  node  A to  C,  but  how 
important  it  is  to  send  messages  from.  A to  C.  This  discus- 
sion dem.onstrates  a workable  methodology  that  is  applicable 
for  the  quantification  of  the  value  of  requirements  and  the 
optimization  of  system,  design.  This  effort,  while  useful  in 
providing  a rough  guide  to  the  importance  of  requirements, 
needs  to  be  extended  in  order  to  arrive  at  a definitive 
statement  of  the  value  of  various  WWMCCS  architectural 
candidates.  Such  a further  extension  must  include  a con- 
sideration of  all  requirements  as  well  as  all  cost  factors. 

The  reader  will  note  that  the  procedure  used  here  was 
an  interval  scale  approach.  In  fact,  at  certain  times, 
ratio  scale  judgments  were  made.  Rather  than  use  indif- 
ference judgments,  the  procedure  used  direct  ratings  by 
setting  a specific  requirement  at  some  value  and  comparing 
others  to  it.  The  reader  will  also  note  that  the  priority 
given  situations  closely  resembles  the  importance  weights 
for  scenarios  discussed  in  Section  3. 

Finally,  the  optimization  done  in  this  example  assumed 
a fixed  budget  that  would  be  expended  on  the  alternatives 
under  consideration.  In  such  a case,  the  use  of  a benefit- 
to-cost  ratio  is  appropriate.  An  important  consideration  in 
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the  general  use  of  such  an  approach  is  that  the  benefit 
scale  should  be  related  to  dollars.  A benef it-to-cost  ratio 
of  100/6.5  million  means  little  until  the  dollar  value  of 
100  units  of  benefit  is  established.  For  example,  if  that 
dollar  value  were  3 million,  the  choice  of  the  alternative 
associated  with  that  ratio  would  be  a poor  one.  If,  as  is 
■he  case  in  the  Kh\'MCCS  example,  the  decision  maker  is  con- 
sidering options  where  the  functions  are  extremely  critical, 
and  all  options  will  lie  in  Region  I of  Figure  4-3,  then 
establishing  the  actual  dollar  value  associated  with  benefit 
nur.bers,  though  advisable,  m.ay  not  be  im.perative.  Obviously, 
assumptions  about  where  options  lie  in  Figure  4-3  should  be 
m.ade  very  carefully. 


Figure  4 3 

BENEFIT  VERSUS  COST  IN  DOLLARS 
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5.0  THE  CHOICE  OF  ONE  OF  SEVERAL  SYSTEMS  CHARACTERIZED 
BY  MULTIPLE  PERFORMANCE  MEASURES 


As  discussed  in  Section  4.6,  the  use  of  cost-to- 
benefit  ratios  is  optimal  as  a criterion  for  choice  given  a 
fixed  budget  constraint  where  the  dollars  are  to  be  allo- 
cated among  many  alternatives.  A different  problem,  is  the 
choice  of  one  of  several  systems,  each  characterized  by 
multiple  performance  m.easures.  Specifically,  the  problem, 

outlined  in  Section  1.0  is  the  following.  The  military  has 

2 

available  N different  C system.s  denoted  as  alternatives  A^ , 

^2^  • • ! Aj^,.  Further,  a set  of  M scenarios  denoted  as  {S^^, 

S^,...,  S,^},  acceptable  given  the  criteria  discussed  in 

Section  3.0,  has  been  established.  For  each  scenario,  an 

acceptable  performiance  measure  has  also  been  established,  and 

the  performance  of  each  system,  in  each  scenario  can  be 

observed.  The  perform.ance  of  system  A^  in  scenario  S^  is 

denoted  as  Also  available  for  each  system,  is  a valid 

projection  of  life  cycle  cost  (or  some  other  acceptable  cost 

estim,ate)  . The  cost  of  system.  A.  is  denoted  as  C..  Given 

1 1 

all  this  inform.ation , which  system  should  the  military 
choose? 


This  description  implies  that  many  of  the  practical 
problems,  discussed  in  earlier  sections  of  this  report,  have 
been  solved.  The  question  here  is  a straightforward  one  of 
applying  decision-analytic  techniques  to  the  choice  of  one 
of  several  options  each  characterized  by  multiple  attributes 
and  cost. 


The  reader  will  note  a similarity  between  this  problem 
and  that  addressed  in  the  example  in  Section  4.0.  In  that 
example,  however,  a fixed  set  of  systems  was  not  available. 
Rather,  a particular  combination  of  capabilities  characterized 
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a hypothetical  system,  and  the  problem  was  one  of  determining 
the  optimial  hypothetical  system  that  could  be  obtained  for 
exactly  the  cost  constraint.  Nonetheless,  the  methodology 
used  for  scaling  benefits,  direct  estimation  of  interval 
scale  value  functions,  is  one  approach  to  establishing  the 
value  of  multiple  attributed  alternatives.  Discussion  of 
that  approach  will  not  be  repeated  here.  Rather,  two  similar 
approaches  to  the  problem  will  be  discussed,  the  first 
characterized  by  stronger  assumptions  concerning  the  ability 
to  trade-off  performance  in  each  scenario  with  dollars  than 
is  the  second. 

5.1  Intra-Scenario  Dollar  Trade-offs 


Each  system.  can  be  considered  to  be  a multiple 
attributed  alternative,  where  the  M + 1 attributes  for  each 
system  are  the  performance  measures  in  each  of  the  scenarios 
and  also  system  cost.  Thus,  can  be  characterized  as: 


A . 
1 


i2' 


11 


iM ' 


C . } . 
1 


It  should  be  noted  that  P.  . is  most  likely  also  a 
multiple  attributed  measure,  an  aggregate  of  several  different 
performance  measures.  That  problem  is  not  addressed  here, 
and  it  is  assumed  that  the  P. . is  a meaningful  measure  with 
respect  to  which  judgments  can  be  made.  It  should  be  noted 
that  if  P^j  is  an  aggregate  other  than  a meaningful  expected 
value,  it  may  be  difficult  to  assess  trade-offs  among  the 
P^j's.  This  practical  problem  merits  attention  and  can  greatly 
increase  the  difficulty  of  the  procedures  to  be  discussed, 
for  both  require  that  the  be  interpretable  measures. 

The  problem  will  be  further  discussed  as  necessary,  but  the 
reader  should  assume  for  purposes  of  discussion  that  the  P^^ 
are  interpretable,  meaningful  measures. 


The  reader  will  also  note  that  "observed"  scenario  per- 
formances are  actually  projections  of  expected  system  perfor- 
m.ance  given  that  the  scenario  actually  occurs  in  the  future. 
The  values  of  these  performances  can  thus  be  weighted  by 
scenario  probabilities  to  yield  an  expected  value. 


The  first  case  to  be  discussed  assumes  that  the  P. . can 

be  directly  translated  into  dollars  in  each  scenario.  A 

possibility  for  would  be  a measure  consisting  of  lives 

and  equipment  lost  because  of  system;  performance,  given  that 

the  scenario  has,  in  fact,  occurred.  In  order  to  use  such  a 

measure,  an  assum.ption  m.ust  be  made  about  a base  cost.  One 

base  to  use  would  be  the  cost  of  the  performance  of  the 

current  system  in  the  scenario.  Assume,  for  purposes  of 

discussion,  chat  all  system.s  achieve  levels  of  perform.ance  in 

each  scenario  superior  to  those  of  the  current  system 

denoted  as  P^^  for  S^.  The  benefit  of  A^  in  is,  then, 

the  dollar  savings  associated  with  the  difference  between 

P.  . and  P ..  Call  this  dollar  savings  B.  ..  (By  definition - 

B =0.)  B..  is  calculated  for  each  A.  in  each  S..  Then, 

CD  ID  ID 

given  that  scenario  probabilities  have  been  correctly  assessed, 

the  expected  benefit  of  deploying  each  A^  can  be  calculated. 

Denote  the  probability  of  as  p^.  Then  the  expected 

benefit  of  deploying  A^,  in  dollars,  is  denoted  as  EB(A^)  = 


rp  ,B , . 

/l  XI. 


Necessary  for  further  calculation  is  the  life  cycle 

cost  of  deploying  the  current  system,  . The  incremental 

cost  of  deploying  A^  is  (C^-C^)=AC^.  The  quantity  (EB(A^)- 

AC^]  is  the  overall  expected  value  of  system  A^  denoted  as 

EV(A.,  measured  in  dollars  (EV (A  )=0  as  defined).  The 
1 c 

system;  with  the  highest  expected  value  is  chosen. 


This  approach  has  the  very  desirable  characteristic 
that  all  benefits  are  equated  to  a meaningful  numeraire,  in 
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this  case,  dollars,  which  enhances  communication  and  helps 
to  ensure  the  validity  of  the  measurement.  As  might  be 
expected,  the  main  difficulty  is  establishing  a performance 
measure  that  can  be  transferred  directly  into  dollars.  The 
measure  chosen  for  this  example  can  be  transformed  into 
dollars  (assum.ing  that  agreement  on  the  dollar  value  of  a 
life  has  been  reached),  but  there  exists  a problem  with  it, 
for  as  defined  it  does  not  necessarily  reflect  the  importance 
of  the  scenario.  In  certain  scenarios,  for  exar;ple,  normal 
patrol  operations,  loss  of  any  lives  or  equipment  is  un- 
acceptable. However,  in  others  like  a general  NATO  war,  the 
loss  of  large  nur.bers  of  lives  and  equipm.ent  will  be  acceptable 
if  an  acceptable  outcome  is  achieved.  In  other  words,  the 
dollar  value  of  lives  and  equipment  lost  may  not  reflect  all 
the  factors  that  determine  the  relative  importance  of  a 
scenario.  Somehow,  the  social  value  for  a life  may  be 
different  for  the  two  cases,  probably  larger  in  the  peace- 
time case.  The  loss  of  a life  is,  most  likely,  best  charac- 
terized as  a m.ultiple  attributed  outcome. 

This  discussion  does  not  condemn  the  use  of  this  pro- 
cedure. Rather,  it  points  out  that  certain  measures,  easily 
translatable  to  dollars,  may  not  at  all  be  reflective  of  the 
decision  maker's  true  preference  function.  The  seemdngly 
sim.ple  answer  is  to  find  an  alternative  measure  that  is. 

For  exam.ple,  a reasonable  substitute  would  be  the  dollar 
value  associated  with  the  difference  in  the  outcomes  for  the 
entire  U.S.  of  deploying  either  the  alternative  under  con- 
sideration or  the  current  system.  However,  such  a loss  is 
reflected  in  loss  of  world  influence,  loss  of  lives  and 
equipment,  expected  future  loss  of  freedom,  loss  of  economic 
power,  loss  of  markets,  and  the  like.  To  translate  such 
measures  into  a dollar  figure  is  obviously  not  an  easy  task, 
but  some  method  of  doing  so  must  be  found  if  the  procedure 
discussed  is  to  be  used.  If  a valid,  meaningful,  performance 
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measure  can  be  found,  one  which  can  be  translated  into 
dollars,  the  procedure  can  be  used.  Section  5.2  discusses 
an  alternative  procedure  that  attempts  to  make  this  trans- 
lation . 

5 . 2 Intra-  and  Inter-Scenario  Subjective  Trade-offs 

The  problem,  will  be  characterized  by  using  the  sam.e  no- 
tation used  in  Section  5.1.  However,  it  will  be  assumed 
that  the  F. . cannot  be  directly  transformed  into  dollars. 
Exam.ples  could  be  certain  simulation  outputs  such  as  the 
expected  numiber  of  enem.y  targets  destroyed,  expert  sub- 
jective ratings  of  the  quality  of  system,  performance  in  each 
scenario,  or  a statistical  aggregate  of  several  perform.ance 
indicators  observed  for  each  system,  in  a scenario.  Whatever 
m.easure  is  chosen,  direct  translation  into  a dollar  value 
that  would  be  an  accurate  reflection  of  the  system,  per- 
form;ance  in  the  scenario  is  not  available. 

i.’ote  that  the  system  performance  measures  in  each 
scenario  can  be  considered  to  be  independent  attributes. 

That  is,  determ.inistic  additive  independence  (DAI),  as 
discussed  in  Section  2.0,  is  likely  to  hold.  This,  of 
course,  is  not  to  say  that  the  system  performance  measures 
are  necessarily  statistically  independent.  Rather,  the 
decision  maker  can  make  judgments  about  the  value  of 
without  considering  the  performance  of  in  the  other 
scenarios.  This  implies  that  if  it  is  desired  to  assess  a 
multiple  attribute  utility  function,  one  of  the  additive 
forms  discussed  in  Section  2.0  is  likely  to  hold.  System, 
values  in  scenarios  can  be  established  by  using  the  method- 
ology discussed,  and  an  expected  system,  value  can  then  be 
calculated . 

Recall  the  earlier  assumption  that  the  performance  of 

the  current  system  A was  inferior  to  that  of  all  other 
•'  c 
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systems  in  each  scenario.  If  the  value  of  achieving  the 

performance  P . ■ in  S . is  denoted  as  U (P . . ) , then  it  is 

assum.ed  that  U(P  .)  - (J(P.  .)  for  all  i,  j.  The  worst  system, 
c]  i;] 

that  could  be  deployed  is  . This  assumption  can,  of 
course,  be  dropped  with  some  added  complication  of  discussion. 

For  each  S^,  there  is  a P. . that  is  superior  to  all 

j 11 

others  in  S-;.  Denote  that  as  P ..  Then  the  value  difference 
^ "1 

for  S . is  U (P  . ) -U (P  .)=!.. 

1 L sj  cjJ  : 

Note  that  P is  the  actual  performance  of  one  of  the 
{A^}  under  consideration.  Accordingly,  is  a relative 
value  difference  for  dependent  on  the  particular  set  of 

options  under  consideration  and  conditional  on  scenario 
occurrence.  Thus,  even  if  a scenario  is  extremely 
critical,  if  the  P^^  for  are  all  nearly  equal,  will 
be  very  low,  probably  less  than  that  of  other  less  critical 
scenarios.  This  result  occurs  because  it  is  assumed  that 
only  the  set  {A^}  under  consideration  is  av'ailable;  that 
is,  the  alternatives  are  fixed.  In  a design  problem,  where 
the  option  of  introducing  a new  system  is  of  major  im.portance, 
a different  upper  bound  on  P^^  would  be  appropriate,  perhaps 
that  of  a hypothetical  perfect  system.  The  approach  using 
the  upper  bounds  of  the  alternatives  under  consideration  is 
a "relative"  one  as  opposed  to  an  "absolute"  approach  using 
so-called  "perfect"  performances.  The  relative  approach  has 
the  advantages  that  the  upper  bound  performance  for  each 
has  been  observed,  and  judgments  with  respect  to  it  are  less 
difficult.  The  "absolute"  approach  allows  the  introduction 
of  alternatives  other  than  the  elements  of  the  set  under 
consideration.  As  indicated,  the  relative  approach  is 
discussed  here. 


The  procedure  to  be  described  involves  first  assessing 
intra-scenario  value  trade-offs.  Then  inter-scenario  value 


trade-offs  can  be  used  to  yield  an  overall  value  measure. 

Consider  scenario  S..  U(P  .)  will  be  set  at  100.  Similarly, 

1 ^ D 

U (P  .)  will  be  set  at  0.0.  The  value  of  A.  for  S.,  U(P  .) 

CD  1 D s] 

is  obtained  by  having  the  decision  maker  assess  the  relative 

position  of  P . . with  respect  to  P . and  P ..  Given  the 
ID  ^ ^ CD  SD 

arbitrary  assessment  such  that  U (P  .)-l?(P  . 

L sD  c: 

value  will  be  some  number  between  0 and  100.  Lottery 

methods  can  be  used  to  obtain  U(P^j).  This  approach  is  an 
Iterative  one  that  asks  the  decision  maker  to  choose  between 
two  options: 


^=100,  the 


Option  0,  - P . . for  certain  S . , and 
1 ID  D 


Option  0^ 


A gamble  in  which  the  decision  maker  will 

receive  P^^  with  some  probability 

otherwise  P • . 

CD 


The  decision  maker  is  asked  to  choose  either  0^  or  0^ , and 
an  indifference  point  is  sought.  When  0^  is  in  fact  observed 
to  be  indifferent  to  0^,  then  given  that  certain  independence 
conditions  earlier  discussed  do,  in  fact,  hold. 


U (P 


ID  ‘ 


= q.  .U{P  .) 
^ID  s: 


) U(P 


CD 


) = lOOq.^ 


All  P. . are  scaled  in  this  manner  for  each 
ID 

M intra-scenario  (or  intra-attribute)  value 
on  a scale  of  0 to  100. 


establishing 
functions,  all 


Note  that  the  q. . can  be  used  to  scale  benefits.  The 

dollar  benefit  value  of  P . can  be  set  at  0.0,  and  the 

CD 

dollar  benefit  value  of  P^^  can  be  assessed  by  using  straight 
subjective  estimations.  Suppose  that  value  can  be  assessed 


as  B 

SD 

q . . B . 
^ID  SD 


Then  the  dollar  benefit  value  of  P^ . would  be 
Given  the  scenario  probabilities,  the  expected 


dollar  benefit  of  A,  would  be  .pq.  B ..  This  would,  of 

1 D D ID  SD 
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course,  require  only  a single  dollar  benefit  estimate  per 

scenario.  The  cost  of  A would  then  be  assumed  to  be  zero, 

c 

and  a procedure  sim.ilar  to  that  in  Section  5.1  would  be 
used.  The  only  difference  is  that  the  benefit  values  are 
straightforward  subjective  estimates.  Of  course,  as  indicated, 
such  estimates  require  that  the  decision  maker  interpret 
the  performance  measure  in  terms  of  dollars. 


Alternatively,  it  could  be  the  case  that  the  decision 
maker  cannot  assign  an  intra-scenario  dollar  equivalent  to 
the  bene  lit  measure  as  described.  Then  it  is  still  possible 
that  the  intra-scenario  benefits  can  be  aggregated  across 
scenarios  to  yield  an  overall  measure  that  can  somehow  be 
traded  off  against  dollars.  It  is  difficult  to  think  of  a 
case  where  intra-scenario  dollar  trade-offs  cannot  be 
assessed  but  where  trade-offs  can  be  assessed  between  an 
aggregate  performance  measure  and  cost.  Nonetheless,  this 
general  problem,  can  be  addressed.  With  all  system.s  scaled 
relatively  within  scenarios,  it  is  necessary  to  aggregate 
values  across  scenarios. 


To  compare  benefits  across  attributes,  a hypothetical 


A is  defined  where  A achieves  P for  each  S.  given  that 
s s sj  j ^ 

S,  has  occurred.  That  is,  A =(P  .,  P _,  ...,  P „) . Recall 
] ^ s si  s2  sM 


that  I j = [u  (Pg j ) -U (P^ j 0 . It  is  necessary  to  establish  the 

relative  values  of  the  1^.  Again,  the  lottery  approach 

will  be  discussed.  Since  a fixed  set  of  alternatives  is 

under  consideration,  only  the  relative  values  of  the  I . 

1 

are  necessary  and  an  arbitrary  0 to  1.0  scale  is  employed. 


With  respect  to  establishing  Ij^,  the  decision  maker  is 
offered  the  following  two  options: 
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OPTION  0^.  The  decision  maker  can  have  a system  which 

performs  at  the  level  of  for  all  scenarios  except  Sj^.  In 

S,  the  system  achieves  P , . That  is, 
k s k 


0,  = A , for  certain  where 
1 sk 


A,  — (P  ,/  P •••/  •••/ 

sk  cl  c2  sk  cM 


OPTION  O^.  The  decision  maker  is  given  a gamble  in 
which  he  gets  A with  probability  w,  , otherwise  A . Again, 

S o 

several  iterations  are  performed  to  achieve  indifference 
between  0^  and  When  that  is  achieved,  with  appropriate 

independence  checks  holding,  it  assures  an  additive  form  of 
the  overall  value  function  and  also  assigns  P^j^  a value  of 
0 and  Pgj^  a value  of  1.0  for  each  and  the  following  is 

true : 


u(0j) 

= U(02)  , 

=U(Pci,Pc2'  •••'  Pgk' 

- U(A^)  = Wj^[U(Ag)  - 

U(A^,) 

- U(A^)  = Wj^(l.O)  , 

But  U(Agj^)  - U(A^)  = Ij^; 

Thus , 

'k  ■ “k- 

The  procedure  guarantees  that 
for  A^  in  is  then 

w. (lOOq. j) , 

and  the  expected  benefit  of  A^ 


U(A^)  ] , 


r 

.w.  = 1.0.  The  relative  benefit 
J J 

is 


EU(A^)  = EP .w.  (lOOq^ .)  . 

4 J J J 
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With  this  aggregate  expected  benefit  for  each  system, 
it  is  necessary  to  consider  trade-offs  between  benefit  and 
cost.  Typically,  when  the  benefit  measure  is  interpretable 
in  terms  of  dollars,  the  procedure  results  in  iso-preference 
contours  such  as  those  displayed  in  Figure  5-1, 


MIN 

FEASIBLE 

H 

C/) 

C 

u 

Co 

Cl 

MAX 
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The  decision  maker  is  asked  questions  of  the  sort: 
suppose  that  for  a cost  of  C^,  you  could  hav-’e  a system  that 
provides  an  expected  benefit  (or  performance)  of  A^;  and 
suppose  performance  could  be  increased  to  and  cost  to 
C.;  which  option  would  you  prefer,  0,  = (A„,C^)  or  0-,=  (A,,C,)? 


EXPECTED  BENEFIT 


Figure  5 1 

COST-BENEFIT  TRADE  OFFS 


Indifference  between  options  is  sought,  and  indifferent 
options  form  iso-preference  contours  as  displayed  in  Figure 
5-1.  Then,  any  system  will  be  represented  as  a point  in  the 
cost/benefit  space,  and  the  system  on  the  iso-preference 
contour  farthest  to  the  right  is  chosen  (assuming  it  is 
af  fordable ) . 

The  difficulty  here  involves  the  nature  of  the  per- 
form,ance  measure.  It  is  an  aggregate  expected  performance, 
where  no  trade-offs  into  dollars  have  been  made.  The  point 
represents  the  expected  benefit  associated  with  deploying 
A^.  Similarly,  the  point  A^  represents  the  expected  benefit 
attainable  if  the  best  performance  could  be  had  in  each 
scenario.  Of  course,  this  point  cannot  occur  given  the 
options  unless  one  system  is  best  in  all  scenarios,  in  which 
case,  there  is  no  decision  problem.  These  two  points  may  be 
interpretable,  but  intermediate  points  miay  be  hard  to 
interpret.  An  A^^  of  50  would  correspond  to  a system,  that  on 
the  average  provides  about  50  percent  of  the  benefit  of  A^. 

Each  system  maps  into  a point  along  the  continuum,,  but  the 
meaning  of  the  points  may  be  very  difficult  to  establish. 

Several  points  along  the  continuum  could  be  given 
meaning  by  the  following  procedure.  Suppose  that  A^  per- 
formis  miserably  in  all  scenarios.  This  result  is  not  un- 
likely, for  it  is  reasonable  to  consider  scenarios  in  which 
the  current  system  is  found  lacking.  For  each  scenario, 
performances  of  the  following  kinds  can  be  defined:  barely 

adequate,  slightly  better  than  adequate,  good,  and  the  like. 
Hypothetical  systems  that  attain  the  same  level  in  each 
scenario  can  be  defined.  These  systems  will  then  achieve 
scores  that  represent  a stable  performance  readily  characterized 
as  barely  adequate,  slightly  better  than  adequate,  good,  and 
the  like.  The  decision  maker  can  most  likely  make  meaningful 
trade-offs  with  respect  to  these  points.  Systems  under 
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consideration  will  then  fall  along  the  continuum  with 
respect  to  these  systems.  Note,  however,  that  the  uncer- 
tainty associated  with  the  systems  under  consideration  will 
be  greater,  for  they  do  not  reliably  achieve  the  same  per- 
formance level  in  each  scenario. 


This  discussion  emphasizes  the  nece 
formance  mieasure  that  can  be  interpreted 
There  is  obviously  no  magic  solution  to 
comparing  cost  and  benefit  where  benefit 
lated  directly  into  dollars.  The  last  a 
can  be  used,  but  it  would  be  better  to  f 
scenario  in  which  cost  and  performance  c 
Given  that  ability,  then  the  trade-offs 
across  all  scenarios  would  allow  transla 
for  all  scenarios. 


ssity  to  find  a per- 
in  terms  of  dollars, 
the  problem,  of 
cannot  be  trans- 
pproach  discussed 
ind  at  least  one 
ould  be  traded  off. 
between  performances 
tion  into  dollars 
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6.0  CONCLUSION 


This  recort  has  covered  many  aspects  of  utility  analysis. 

2 

The  question  of  "how  much  C is  enough?"  has  been  addressed 
at  a fairly  abstract  methodological  level.  Emphasis  has 
bee.n  on  the  methodology  that  might  be  used  to  answer  the 
question.  As  indicated  in  Section  1.0,  the  question,  though 
appearing  to  be  an  absolute  one,  is  necessarily  relative, 
involving  multi-criteria  trade-offs.  Most  of  the  report  has 
dealt  with  the  methodology  for  making  such  trade-offs.  That 
m.ethodology  is  well  established,  and,  as  exem.plified  in 
Section  4.0,  it  has  been  applied  repeatedly  to  practical 
prcblem.s  . 

Section  5.0  has  specifically  addressed  trade-offs 
between  cost  and  multiple  performance  measures.  The  dis- 
cussion has  emphasized  the  necessity  for  somehow  establishing 
a benefit  measure  that  can  be  interpreted  with  respect  to 
dollars.  As  discussed,  this  can  be  accomplished  by  direct 
translation  of  performance  to  dollars  or  by  subjective 
trade-offs  between  levels  of  an  aggregate  performance  measure 
and  dollars.  As  indicated  in  the  discussion,  the  more 
directly  interpretable  the  perform.ance  measure  in  terms  of 
dollars,  the  less  difficult  the  trade-offs  between  cost  and 
performance . 
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APPENDIX: 


RESULTS  OF  THE  U’KMCCS  ANALYSIS 


The  following  pages  contain  the  results  of  the  initial 
attenpt  to  define  requirement  levels  and  to  evaluate  the 
levels  in  a consistent  and  comparable  manner.  These  level 
definitions  and  valuations  are  to  be  interpreted  as  a guide 
to  the  im.portance  of  meeting  requirements  to  varying  degrees 
(It  was  concluded  in  the  study  that  extension  of  the  results 
to  the  justification  of  architectural  engineering  candidates 
for  WI-.'XCCS  systems  could  require  redefinition  or  refinem.ent 
of  perform.ance  levels  in  some  cases.)  The  meaning  of  estab- 
lishing these  importance  values  is  presented  in  Section  4.3. 

WV7MCCS  CRISIS  REQUIREMENT  LEVELS 


Relative 

Im.portance 

Value 


6.7 


6.0 


3.4 


2.3 


0 


3.2.1  NCA  - Heads  of  State  Interface 

V Secure  point  to  point  voice;  secure 

high-speed  text;  near  simultaneous  voice 
and  text  translation;  with  14  countries 
(understandable  throughput  in  about  5 
minutes) . 

IV  Secure  voice;  secure  high-speed  text; 

manual  on  call  translation;  14  countries 
(system  activation  in  5 m.inutes/under- 
standable  throughput  in  30  to  45  minutes) . 

■ II  IV  above,  with  8 countries  (system. 

activation  in  5 minutes/  20  to  45  minutes 
understandable  throughput) . 

II  Secure  voice  only  with  8 countries 

Manual  on  call  translation.  (15  to  30 
minutes  for  system  activation) . 

I Current  capability  - voice  and  TWX  to 
two  capitals.  Otherwise  through  State/ 
embassy  channels. 
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3.2.2  NCA  - WA\’MCCS  Interface 


8 . 4 


5.0 

0 


8.9 


8.0 


4.4 


0 


6.2 


4.6 


3.1 


III  Secure  voice  with  video;  visual/photo 
transmission;  graphics  display  both 
fixed  and  mobile. 

II  Secure  voice  and  graphics  (data)  only; 
fixed  and  mobile. 

I Current  system  mobile  (secure  in  most 
modes)  voice;  text  to  White  House  situation 
room;  intelligence  net  to  situation 

roomi;  LDX  to  situation  room,  from  the 
NMC . 

3.2.3  WV.’.MCCS  Washington  Level  Interagency  Interface 

IV  Secure,  dedicated  multi-node  voice  with 
video;  message  generation  capability; 
visual/photo  transmission  rate  of  15  to 
20  photos  per  hour  among  the  MNCC  and 
control  rooms  of  approximately  15  departments/ 
agencies . 

III  Above,  except  no  video  and  5 agencies 
instead  of  15. 

II  Secure,  dedicated  multi-node  voice  plus 
text  and  photo  transm.ission  among  5 
nodes . 

I Current  system  - point  to  point  secure 
voice  and  intelligence  warning  net;  LDX 
among  White  House  situation  room,  N.MCC, 

CIA,  and  State. 

3.2.4  Vertical-Horizontal  Analysis  Interface  for 
Command  Post  Duty  Personnel  (Operations, 
Intelligence,  Logistics  Analyst) 

IV  Secure,  multi-node  voice;  multi-node  on- 
line text  transmission;  and  multi-node 
visual/photo  at  15  to  20  per  hour  rate; 
vertically  and  (CINC  to  CINC)  horizontally 
to  include  transportable  C^. 

III  Above,  but  without  horizontal  CINC  to 
CINC  capability  and  CINC  to  transportable 
C3. 

II  Secure  multi-node  voice  and  on-line  text 
in  vertical  mode  only. 
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0 


10.8 


9.7 


8.1 


4 . 9 


0 


4.5 


2.2 


0 


I Current  systeir.  - point  to  point  secure 
(not  dedicated)  voice  and  text. 

3.2.5  WWMCCS  CoiTununication  for  Senior  Decision 
Makers 

V Secure  ir.ulti-node  voice;  visual/photo 
transmission  at  15  to  20  per  hour  on- 
line message  generation  among  NMCC, 
established  U and  S command  and  control 
facilities,  JTF , NATO  and  four  or  five 
selected  Allies. 

IV  Secure  multi-node  voice;  visual/photo 

transmission  and  on-line  text  transmission 
among  NMCC,  U and  S commiands , JTF  and 
NATO . 

Ill  Secure  multi-node  voice;  visual/photo 
transmission;  on-line  text  among  NMCC 
and  U and  S comm.ands;  same  except  no 
visuai/{.'hoto  for  JTF  and  NATO. 

II  Secure  multi-node  voice  with  instant 
activation  and  on-line  text  am.ong  NMCC, 

U and  S comjr,ands,  JTF  and  NATO. 

I Current  system,  - multi-node  voice  and 
text  transm.ission  among  NMCC,  U and  S 
commands,  JTF  and  NATO. 

3.2.6  In’WMCCS  Fixed  Site  Allied  Interface  (a 
small,  highly  transportable  capability  for 
augmienting  norm.al  embassy  facilities) 

III  Secure  point  to  point  voice;  high-speed 
text  transmiission;  photo/visual  transm.ission 
capability  at  rate  of  15  to  20  per  hour. 
Manual  language  translation;  capability 

to  operate  at  almost  any  embassy  location. 
Capability  to  connect  with  nearest  CINC, 
as  well  as  NMCC.  Deploy  time  one  to  two 
v.’ays . 

II  Same  general  requirements  as  III,  with 
more  limited  linguistic  capability  and 
operating  area.  Capability  to  connect 
with  NMCC  only. 

I Current  system  - standard  embassy  facilities. 
Almost  no  secure  voice;  limited  text 
facilities . 
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2.8 


2 . 2 


0 


5.8 


5.2 


2.6 

0 


8.8 


3.2.7  Transportable  C Facility  {An  overall 
capability  generally  comparable  to  a CINC- 
level  facility) 

III  Secure  multi-node  voice;  on-line  message 
generation;  visual/photo;  ADP  storage- 
retrieval  sys*-em;  connection  with  NMCC 
and  CINCS.  En  route  up-date  capability 
for  operation  in  air,  sea,  and  land  m.ode 
(for  use  in  situations  when  perm.anent 
facilities  as  in  3.2.5  do  not  exist). 

Deploy  time  one  to  two  days. 

II  Same  as  III  above,  except  transportable 
by  sea  and  air;  operates  in  land-based 
mode  only. 

I Current  system.  - none,  except  such 
developmental  systems  and  such  mobile 
facilities  as  are  inherent  in  comm.and- 
control  type  aircraft  and  ships. 

3.2.8  Monitoring  Foreign  National  Behavior 

IV'  Stored,  crises  oriented  data  on  the 
behavior  patterns/characteristics  of 
approximiately  30  foreign  nations,  with 
rapid  up-date  capability  for  access  and 
display  (within  5 minutes)  at  NMCC,  the 
CINCS,  the  JTF , and  selected  Washington 
agencies  and  departments  (i.e.,  CIA, 

State ) . 

III  Same  as  IV  above,  except  with  access  and 
display  at  NMCC,  W'hite  House  situation 
room.,  CIA  and  State. 

II  Sam.e  as  III  above,  except  with  access 
and  display  only  at  NMCC. 

I Current  system  - m.anual  data  handling; 
incomplete  data. 

3.2.3  Intelligence  Interface 

III  A direct,  NMCC  interface  with  all  major 
intelligence  agencies,  with  some  capability 
to  access  and  display  both  stored  and 
incoming  intelligence  from  a variety  of 
sources  and  to  task  those  sources  directly 
for  specific  information  in  near  real- 
time . 


»T 
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II 


6 . 2 


0 


7.6 


6.1 


0 


8.1 


An  interface  with  NMIC  and  reliance  on 
the  NMIC  worldwide  intelligence  systerr, 
for  automated  intelligence  displays  in 
the  NMCC  and  for  tasking  DoD  and  certain 
other  intelligence  resources  through  the 
NMIC  in  support  of  VAv"MCCS. 


I Current,  entirely  manual  system.;  mostly 
hard  copy;  some  direct  access  to  files 
of  other  agencies  through  NMICC. 

3.2.10  Ad  Hoc  Planning 


3.2.11  User  Oriented  ADP-Based  Support 


III  An  audio-visual  display/query  capability 
specifically  oriented  to  the  specific 
needs  of  NCA,  CJCS,  the  U and  S commanders 
and  other  high-level  decision  makers 
desiring  direct  access  to  synthesized, 
yet  traceable,  day  to  day  and  crises 
managemient  problem-solving  situations. 

II  Sam.e  as  III  above,  but  access  lim.ited  to 
Washington  area  decision  makers,  i.e., 
CJCS,  Secretary  of  Defense,  NCA. 

I Current  system,  which  is  alm.ost  completely 
mianual,  responds  in  hours,  not  in  minutes. 


3.2.12  Integrated  Data  Display 

3.2.13  Dynamic  Situation  “assessment 

III  NMCC  crisis  team,  access  to  V\T7.MCCS  intelli- 
gence, operations  and  logistics  data 
files  with  near  real-tim.e  capability  to 
integrate  data  and  display  it  in  the 
form  of  situation  summaries,  executive 
up-dates  and  optional  courses  of  action 
to  NCA,  CJCS,  the  State  and  CIA  control 
centers  and  selected  CINCS  (when  required) 
This  includes  a capability  for  the 
crisis  team  to  display  source  data  from 
other  agency  and  U and  S command  files. 
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rr 


4.9 


II 


0 


10.1 


5.4 


0 


3.4 


2.6 

1.0 


Some  as  III  above,  except  that  integrated 
displays  are  not  available  to  U and  S 
commands,  the  JTF , State  and  CIA;  and 
information  from  files  outside  of  V.VMCCS 
cannot  be  accessed  and  displayed  by 
crisis  teams  in  N.MCC  (an  internal  NMCC 
system) . 

I Current  system  with  lim.ited  display 
capability.  Almost  all  of  the  integration 
process  is  manual.  Reaction  times  are 

in  hours,  not  minutes. 

3.2.14  Crises  Monitoring  Support 

III  An  audio-visual  display/query  crises 
monitoring  system,  continuously  available 
to  the  NCA,  Secretary  of  Defense,  CJCS, 
and  other  high-level  officials  that 
provide  the  status  of  crises  actions  and 
results  continuously  as  the  situation 
changes  and  whenever  the  system,  is 
queried.  System,  is  decision  malcer 
oriented  and  provides  analysis  support 
during  "day  to  day"  situations. 

II  Same  as  III  above,  except  system,  is  up- 
dated periodically  (norm.ally  4 to  6 hour 
up-dates)  as  opposed  to  continuously  in 
III  above. 

I Secure  voice,  message,  courier,  etc. 
from  MMCC  to  decision  m.a)cers. 

3.2.15  ADP-based  Standardized  Operating  Procedures 

IV  An  advanced  ADP-based  system  for  alerting 
personnel  of  a crisis  (or  impending 
crises),  initiating  the  data  file  search, 
generating  reports  on  the  status  of  the 
data  base  on  the  area  of  interest, 
identifying  tasl<ing  requirements,  initializing 
data  for  analysis  models  and  other 
automated  problem-solving  support; 
available  to  NMCC,  NCA,  U and  S comjr.and, 

and  JTF. 

III  Same  as  IV  above,  but  available  to  NMCC 
only . 

II  ADP-based  print-outs  and  graphic  displays 
of  SOPS  and  the  location  of  )<ey  personnel; 
available  to  NMCC  only. 
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0 I Current  system,  which  has  almost  entirely 

manual  checklists  and  educators,  somie 
lim.ited  off-line  modeling. 

3.2.16  Autom.ated  Crises  Information  Directory 

3.5  III  Automated  system  for  displaying  location 

and  status  of  inform.ation  files  and 
knowledgeable  substantive  experts  within/ 
outside  the  DoD  to  NMCC  and  the  U and  S 
comjnands  with  the  directory  inform.ation 
available  also  to  selected  Washington 
agencies . 

2.8  II  Same  as  III  above,  with  access  and 

display  for  NMCC  only. 

0 I Manual  containing  lim.ited  number  of 

files /personnel . 

3.2.19  Intelligence/Operations/Diplomatic  Coordination 

4.5  IV  Voice  conferencing  within  each  area  or 

community;  point  to  point  message;  photo 
transm.ission  am.ong  elements  (operations; 
intelligence,  diplomatic)  within  area. 

3.8  III  Same  as  IV  above,  except  no  photo  transmission. 

2.7  II  Secure  conference  voice  only  at  the 

above  locations  (IV) . 

0 I Current  system.,  which  does  not  have 

dedicated  facilities  at  most  locations. 


REFERENCES 


An  Analytic  Procedure  for  Examining  the  Relation  Between 
W^n'NCCS  Performance  Capabilities  and  Funding  Levels. 

Report  prepared  for  the  IBM  Corporation  in  support  of 
U.S.A.F.  Contract  F19628-74-C-0158 . McLean,  VA.; 
Decisions  and  Designs,  Inc.,  March,  1975. 

Edwards,  W.,  and  Tversky,  A.,  eds..  Decision  Making. 

Middlesex,  England:  Penguin  Books,  1967. 

Keel in,  T.W.,  III,  A Protocol  and  Procedure  for  Assessing 

Multi-Attribute  Preference  Functions"  Ph.D.  Dissertat ion 
Stanford  University,  September,  1976. 

Keeney,  L.,  and  Raiffa,  H.  Decisions  with  Multiple  Objectives 
Preferences  and  Value  Tradeoff s.  New  York:  John" Ki lev . 

TWT.  ~ 

O'Connor,  M.F.,  and  Edwards,  W.  On  Using  Scenarios  in  the 
Evaluation  of  Complex  Alternatives.'  Technical  Report 
TR  76-17.  McLean,  VA.  : Decisions  and  Designs,  Inc., 

December,  1976. 

Raiffa,  H,  Decision  Analysis:  Introductory  Lectures  on 

Choices  under  Uncertainties.  Reading,  MA. : Wesley- 

Wesley , 1968 . 

Rumsfeld,  D.H.  FY  1977  Report  of  Secretary  of  Defense. 

United  States  Department  of  Defense,  1$76 . 


75 


CONTRACT  DISTRIBUTION  LIST 
(Unclassified  Technical  Reports) 


Director 

Advanced  Research  Projects  Agency 
Attention;  Progratr,  Management  Office 
lAOO  Wilson  Boulevard 
Arlington,  Virginia  22209 

Office  of  Naval  Research 
Attention:  Code  455 

800  North  Quincy  Street 
Arlington,  Virginia  22217 

Defense  Documentation  Center 
Attention:  DDC-TC 

Cameron  Station 
Alexandria,  Virginia  22314 

DCASMA  Baltimore  Office 
Attention:  Mr.  K.  Gerasim 

300  East  Joppa  Road 
Towson,  Maryland  21204 

Director 

Naval  Research  Laboratory 
Attention:  Code  2627 

Washington,  D.C.  20375 

Office  of  Naval  Research 
Attention:  Code  102IP 

800  North  Quincy  Street 
Arlington,  Virginia  22217 


2 copies 


3 copies 


12  copies 


1 copy 


6 copies 


6 copies 


76 


SUPPLEMENTAL  DISTRIBUTION  LIST 
(Unclassified  Technical  Reports) 

Department  of  Defense 


Director  of  Net  Assessment 
Office  of  the  Secretary  of  Defense 
Attention:  MAJ  Robert  G.  Gough,  USAF 

The  Pentagon,  Room  3A930 
Washington,  DC  20301 

Assistant  Director  (Net  Technical  Assessment) 
Office  of  the  Deputy  Director  of  Defense 
Research  and  Engineering  (Test  and 
Evaluation) 

The  Pentagon,  Room  3C125 
Washington,  DC  20301 

Assistant  Director  (Environmental  and  Life 
Sciences) 

Office  of  the  Deputy  Director  of  Defense 
Research  and  Engineering  (Research  and 
Advanced  Technology) 

Attention;  COL  Henry  L.  Taylor 
The  Pentagon,  Room  3D129 
Washington,  DC  20301 

Director,  Defense  Advanced  Research 
Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Cybernetics  Technology  Office 
Defense  Advanced  Research  Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  ARPA  Regional  Office  (Europe) 
Headquarters,  U.S.  European  Command 
APO  New  York  09128 

Director,  ARPA  Regional  Office  (Pacific) 

Staff  CINCPAC,  Box  13 
APO  San  Francisco  96610 

Dr.  Don  Hirta 

Defense  Systems  Management  School 

Building  202 

Ft.  Belvolr,  VA  22060 


Chairman,  Department  of  Curriculum 
Development 
National  War  College 
Ft.  McNair,  4th  and  P Streets,  SW 
Washington,  DC  20319 

Defense  Intelligence  School 
Attention;  Professor  Douglas  E.  Hunter 
Washington,  DC  20374 

Command  and  Control  Technical  Center 
Defense  Communications  Agency 
Attention:  Mr.  John  D.  Hwang 

Washington,  DC  20301 


Department  of  the  Navy 

Office  of  the  Chief  of  Naval  Operations 
(OP-987) 

Attention:  Dr.  Robert  G.  Sm.ith 

Washington,  DC  20350 

Office  of  the  Chief  of  Naval  Operations 
(OP-951) 

Washington,  DC  20450 

Office  of  Naval  Research 

Assistant  Chief  for  Technology  (Code  200) 

800  N.  Quincy  Street 

Arlington,  VA  22217 

Office  of  Naval  Research  (Code  230) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 

Naval  Analysis  Programs  (Code  431) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Operations  Research  Programs  (Code  434) 
800  North  Quincy  Street 
Arlington,  VA  22217 


Office  of  Naval  Research  (Code  436) 
Attention:  Dr.  Bruce  MacDonald 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Information  Systems  Program  (Code  437) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Director,  ONR  Bra»ch  Office 
Attention:  Dr.  Charles  Davis 

536  South  Clark  Street 
Chicago,  IL  60605 

Director,  ONR  Branch  Office 
Attention:  Dr.  J.  Lester 

495  Summer  Street 
Boston,  MA  02210 

Director,  ONR  Branch  Office 
Attention:  Dr.  E.  Gloye 

1030  East  Green  Street 
Pasadena,  CA  91106 

Director,  ONR  Branch  Office 
Attention:  Mr.  R.  Lawson 

1030  East  Green  Street 
Pasadena,  CA  91106 

Office  of  Naval  Research 
Scientific  Liaison  Group 
Attention:  Dr.  M.  Bertin 

American  Em.bassy  - Room  A-407 
APO  San  Francisco  96503 

Dr.  A.  L.  Slafkosky 
Scientific  Advisor 

Commandant  of  the  Marine  Corps  (Code  RD-1) 
Washington,  DC  20380 

Headquarters,  Naval  Material  Command 
(Code  0331) 

Attention:  Dr.  Heber  G.  Moore 

Washington,  DC  20360 


Naval  Personnel  Research  and  Development 
Center 

Acquisition  and  Initial  Service  (Code  310) 
Attention:  Dr.  Leonard  P.  Kroeker 

San  Diego,  CA  92152 

Naval  Personnel  Research  and  Development 
Center  (Code  305) 

Attention:  LCDR  O'Bar 

San  Diego,  CA  92152 

Navy  Personnel  Research  and  Development 
Center 

Manned  Systemis  Design  (Code  311) 

Attention:  Dr.  Fred  Muckier 

San  Diego,  CA  92152 

Naval  Training  Equipment  Center 
Human  Factors  Department  (Code  N215) 
Orlando,  FL  32813 

Naval  Training  Equipment  Center 
Training  Analysis  and  Evaluation  Group 
(Code  N-OOT) 

Attention:  Dr.  Alfred  F.  Smode 

Orlando,  FL  32813 

Director,  Center  for  Advanced  Research 
Naval  War  College 
Attention:  Professor  C.  Lewis 

Newport,  RI  02840 

Naval  Research  Laboratory 

Communications  Sciences  Division  (Code  540i 
Attention:  Dr.  John  Shore 

Washington,  DC  20375 

Dean  of  the  Academic  Departments 
U.S.  Naval  Academy 
Annapolis,  MD  21402 

Chief,  Intelligence  Division 
Marine  Corps  Development  Center 
Quantico,  VA  22134 


neaa,  tiuman  ractors  Division  Department  of  the  Army 

Naval  Electronics  Laboratory  Center 

Attention:  Mr.  Richard  Coburn  Deputy  Under  Secretary  of  the  Army 

San  Diego,  CA  92152  (Operations  Research) 

The  Pentagon,  Room  2E621 

Dean  of  Research  Administration  Washington,  DC  20310 

Naval  Postgraduate  School 
Attention:  Patrick  C.  Parker 

Monterey,  CA  93940 


78 


Director,  Army  Library 
Army  Studies  (ASDIRS) 

The  Pentagon,  Room  1A534 
Washington,  DC  20310 

U.S.  Army  Research  Institute 
Organizations  and  Systems  Research  Laboratory 
Attention:  Dr.  Edgar  M.  Johnson  (PERI-OK) 

1300  Wilson  Boulevard 
Arlington,  VA  22209 

Director,  Organizations  and  Systems 
Research  Laboratory 
U.S.  Army  Institute  for  the  Behavioral 
and  Social  Sciences 
1300  Wilson  Boulevard 
Arlington,  VA  22209 

Technical  Director,  U.S.  Army  Concepts 
Analysis  Agency 
8120  Woodmont  Avenue 
Bethesda,  MD  20014 

Director,  Strategic  Studies  Institute 
U.S.  Army  Combat  Developments  Command 
Carlisle  Barracks,  PA  17013 

Commandant,  Army  Logistics  Management  Center 
Attention:  DRXMC-LS-SCAD  (ORSA) 

Ft.  Lee,  VA  23801 

Department  of  Engineering 
United  States  Military  Academy 
Attention:  COL  A.  F.  Grum 

West  Point,  NY  10996 

Chief,  Studies  and  Analysis  Office 
Headquarters,  Army  Training  and  Doctrine 
Command 

Ft.  Monroe,  VA  23351 

Commander,  U.S.  Army  Research  Office  (Durham.) 
Box  CM,  Duke  Station 
Durham,  NC  27706 


Department  of  the  Air  Force 

Assistant  for  Requirements  Development 
and  Acquisition  Programs 
Office  of  the  Deputy  Chief  of  Staff  for 
Research  and  Development 
The  Pentagon,  Room  4C331 
Washington,  DC  20330 


Air  Force  Office  of  Scientific  Research 
Life  Sciences  Directorate 
Building  410,  Bolling  AFB 
Washington,  DC  20332 

Commandant,  Al’"  University 
Maxvell  AFB,  Al  36112 

Chief,  Systems  Effectiveness  Branch 
Human  Engineering  Division 
Attention:  Dr.  Donald  A.  Topmlller 

Wrlght-Patterson  AJB,  OH  45433 

Director,  Advanced  Systems  Division 
(AFHRL/AS) 

Attention:  Dr.  Gordon  Eckstrand 

Wright-Patterson  AFB,  OH  45433 

Commander,  Rome  Air  Development  Center 
Attention:  Mr.  John  Atkinson 

Griffis  AFB 
Rome,  NT  13440 

IRD,  Rome  Air  Development  Center 
Attention;  Mr.  Frederic  A.  Dion 
Griffis  AFB 
Rome,  NT  13440 

HQS  Tactical  Air  Command 
Attention:  LTCOL  David  Dianich 

Langley  AFB,  VA  23665 


Other  Government  Agencies 

Chief,  Strategic  Evaluation  Center 
Central  Intelligence  Agency 
Headquarters,  Room  2G24 
Washington,  DC  20505 

Director,  Center  for  the  Study  of 
Intelligence 

Central  Intelligence  Agency 
Attention:  Mr.  Dean  Moor 

Washington,  DC  20505 

Office  of  Life  Sciences 
Headquarters,  National  Aeronautics  and 
Space  Administration 
Attention:  Dr.  Stanley  Deutsch 

600  Independence  Avenue 
Washington,  DC  20546 


79 


*»  • 


Other  Institutions 


Institute  for  Defense  Analyses 
Attention:  Dr.  Jesse  Orlansky 

400  Army  Navy  Drive 
Arlington,  VA  22202 

Director,  Social  Science  Research  Institute 
University  of  Southern  California 
Attention;  Dr.  Ward  Edwards 
Los  Angeles,  CA  90007 

Professor  Ra\'Tnond  Tanter 
Political  Science  Department 
The  University  of  Michigan 
•Ann  Arbor,  Michigan  48109 


80 


i:\r.T. AS?;TFTf.n 


security  Cl  »SSIFIC*TI0N  of  this  page  r»Ti«n  Dmim  Fnltmd 


REPORT  tX)CUMENTATION  PAGE 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


' RtPQR''  NUMBER 

TR  -77-'4 


|2  GOVT  ACCESSION  NO 


3 RECIPiEN  T'S  C AT  Ai_OG  NUMBE  B 


4 T|TL  F '••mf 

PROCEDURES  FOR  ASSESSING  THE  VALUE  OF  COMMAND 
AND  CONTROL  CAPABILITIES  ■ 


5 Type  of  report  s perioo  covered 


/TECHNICAL 


6 RtBFORMtNG  T’RG  REPORT  NUMBER 


T.  AUTHOR'** 


Michael  F.  O'Connor 


-7 


8 contract  OO  GRAn*^  number 


N00014-76-C-0074 


10  ' P»ROGB/fM  TLEMCN*^  TASx- 

AREA  ft  UNIT  NUMBEBS 


9 perform  n G OHG  ani  2 a T ion  name  AnD  ADDRESS 

Decisions  and  Designs,  Incorporated 
Suite  600,  8400  Westpark  Drive 
McLean,  Virginia  22101 


II  controlling  office  name  and  address 

Defense  Advanced  Research  Projects  Agency 
1400  Wilson  Boulevard 
Arlington,  Virginia  22209 


12,  REPORT  DATE 

July  19:7 


13  NUMB  E R O ^ R AGES 


92 


'4  MON  ^^QR  nG  agency  name  ft  ADORES S'’</  diffTTit  frorr  Controlling  OUiC9' 

Office  of  Naval  Research 
800  North  Quincy  Street 
Arlington,  Virginia  22217 


15  security  Cl  ass  iBf  imt  rtrotr  ' 


L^’CLASSIFIED 


I5«  declassification  DOanGRADiNG 
SCHEDULE 


16  DiS*^  Ri  BuTiON  S'^a^EmEnT  (of  thl»  R»port) 

Approved  for  public  release;  distribution  unlimited. 


17  Distribution  statement  (oI  ah$irmn  »nfr»d  In  Block  20.  H dlflormnt  frorr  Roport 


18  SuPRlEmEnTARY  notes 


19  KEY  WORDS  fContInv  on  f*vr»0  mide  it  nocmutmry  and  tdanUfy  by  block  numbor) 

2 

command  and  control  (C  ) function 


utility  analysis 
ordinal  approach 
interval  scale  approach 
multiple-attributed  alternatives 


discriminative  scenario 
representative  scenario 
performance  capability 
value  assessment  methodology 
design  optimization  


20  abstract  rConilnuo  on 


• • »ld*  If  nmc»00mry  and  Idantlfy  by  block  numbar) 

2 
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19.  evaluation  methodology 
cost-benefit  ratio 


weighed  against  the  potential  benefits  of  allocating  funds  elsewhere. 
Examination  of  the  breakdown  of  the  Department  of  Defense  Budget  would 
prompt  a reasonable  inquiry:  Is  this  allocation  scheme  the  best  overall 

scheme  for  meeting  the  long-term  goal  of  defending  the  nation?  More 
specifically,  is  the  seven-percent  allocation  for  sufficient  to  meet 
the  need  in  that  area?  The  answer  necessarily  Involves  assessing  the 
requlrem.ents  for  and  the  benfits  to  be  dervled  from  expenditures  in 
each  of  the  specified  DoD  program  areas.  The  procedures  and  recommenda- 
tions discussed  with  respect  to  generalize  trade-offs  among  objectives 
on  goals  at  any  level  of  generality  chosen  for  analysis. w 

The  objective,  then,  is  to  determine,  given  the  fixed  budget  size, 
whether  the  chosen  dollar  allocations  are  optimal.  From  a decision 
analysis  point  of  view,  this  problem  Involves  assigning  a value  to  the 
action  of  deciding  to  deploy  a particular  system  in  an  uncertain 
future.  A large  decision  tree  would  be  constructed  to  accommodate  all 
potentially  reasonable  deployments,  the  uncertain  events,  and  the 
possible  outcomes.  The  problem  in  such  a case  is  that  the  decision  tree 
is  too  large  ("a  very  bushy  mess,"  to  borrow  a Howard  Raiffa  metaphor), 
and  meaningful  probabilities  and  values  cannot  be  assessed. 

The  discussion  of  the  problem  is  divided  into  four  parts.  Section 
2,0  deals  with  the  theory  of  assessing  value  for  multiple  attributed 
alternatives.  Section  3.0  discusses  the  use  of  scenarios  as  a solution 
to  the  so  called  "bushy  mess"  problem.  Section  4.0  Illustrates  the 
application  of  value  assessment  procedures  to  a hypothetical  problem  of 
evaluating  alternative  architectural  candidates  for  a World  Wide  Military 
ConuTiand  and  Control  System  (WWMCCS)  . Section  5.0  further  Illustrates 
the  application  of  utility  assessment  procedures  to  the  problem  by 
using  a specific  problem:  the  trade-offs  between  cost  and  multiple 

performance  measures. 

This  report  covers  m.any  aspects  of  utility  analysis.  The  question 
of  "how  much  C2  is  enough?"  is  addressed  at  a fairly  abstract  methodologi- 
cal level,  and  emphasis  is  on  the  methodology  that  might  be  used  to 
answer  it. 
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